Cameras in Machine Vision – Part 2 . . . . by Nello Zuech, President, Contributing Editor

As suggested in Part 1 of this article, cameras or the ‘‘eyes’‘ of a machine vision system have come a long way. When I first became involved in the early days of machine vision the standard camera was a vidicon. These truly all analog cameras left a lot to be desired – sensitivity varied pretty much indiscriminately +/- 30%, as fields changed with temperature during the day so, too, did position of readout. To assure consistency in performance required a TV technician – hardly a cost-effective substitution for a direct labor inspector.

To catch up with the latest activities in machine vision related cameras, input for this article was canvassed from all known suppliers of cameras in the machine vision market. Because we received as many responses as we did, we reduced the answers into two papers. This paper looks at the answers to the last six questions asked.

In addition to providing further insight into application issues surrounding machine vision and providing tips to be successful, Part 2 of this article examines the trends that are observed that will impact the cameras and machine vision of cameras.

The following contributed to this article:
Dave Gilblom – President, Alternative Vision
Christophe Robinet – Marketing Manager, Cameras, Atmel
Dr. Joachim Linkemann – Product Manager, Basler Vision Components
Dave Lane – Senior Application Engineer and Jin Kwon – Sales Engineer, Cohu Electronics
Steve Daicos - President, The COOKE Corporation and Dr. Gerhard Holst – Head of Research Department, PCO AG
Bill Laraway – Sales and Marketing Manager, Daitron
David Cochrane – Director of Product Mgmt and Marketing, Vision for Machines, Dalsa Corporation
Loren Ulrich – Marketing Manager, Imperx
Gunnar Jonson – Director of Product Marketing, JAI Group
Dr. Bernhard Mandel – Director of Sales & Marketing, Camera Business, Photonfocus AG
Marty Furse – President, Prosilica
Keith Russell – Director of Marketing, Redlake
Ilias Levis – Product Manager, Sony Visual Imaging Products
John Merva – President, Tattile, USA

5. What is it that you require from a prospective camera buyer for a machine vision application to assure you will deliver the proper camera?

[John] A complete spec for the application is always desirable but not always available.  With Gig E and Camera Link applications many technical details must be exchanged and the customer must be technically savvy.  This is not always the case and requires the vendor to be extremely customer service oriented.

[Ilias] Some customers are very good in providing manufacturers with a detailed requirement list for their application. At a minimum, specifications such as acquisition speed, trigger scheme, resolution and form factor are necessary in order to determine the right camera model for the right application.

[Keith] Fundamentally the camera buyer should understand their application requirements, especially the minimum defect size and required processing rate.  Based on this information a camera supplier can recommend a camera with enough resolution and a fast enough frame rate.

[Marty] The first information that we require from a prospective camera buyer is information about their application.  After understanding the customer’s application requirements we can guide them in their vision-related engineering decisions including the choice of a suitable camera.   Information related to functional requirements, optical considerations, environmental considerations, computer issues, and illumination details, are all helpful in guiding a customer through machine vision choices.

[Bernhard] We require application information, volume of project, references.

[Gunnar] It is important to have an open dialogue about the customer's application. As this sometimes requires getting knowledge about the customer's proprietary methods, a non-disclosure agreement (NDA) should always be established prior to detailed discussions. Without completely understanding the customer’s application, it is hard to translate their request into specifying a specific camera.

[Loren]  The first thing we need to know is the obvious:  A.  Sensor Type (CMOS, Full Frame CCD, Interline Transfer CCD, Etc.), B.  Resolution and C. Frame rate.  Other information that is important to know:

D.  Illumination.  Does the illumination meet the spectral response curve of the sensor (UV, visible, NIR)?
E.  Dynamic Range.  Does the camera exceed their dynamic range requirements?  Is the signal-to-noise ratio sufficient? On the other side of the spectrum, do they need less of a camera than we have to offer?
F.  Integration Issues.  Does the camera need to be triggered (hard trigger, software, PIV). Do multiple cameras need to be synchronized? 
G.  Environment.  Vibration (enough illumination to allow for short exposure interval to eliminate/minimize vibes), temperature. Etc.

[Bill] Thorough information and specifications on the application are necessary to choosing the right camera.  Educated buyers knowing what features they require do not always happen, so we need to know as much about the application as possible to help in choosing the proper camera.  In addition, helping the customer in determining the optimal peripherals to complement the camera is also quite helpful.  As important as a camera is to a successful imaging system, lensing and lighting are paramount in machine vision system designing.

[David C.] If customers can provide us with a basic understanding of their application, cost targets, speed requirements, throughput targets, image resolution, image quality requirements and the size of the defect they are looking for, we can provide the camera that best suits their application.

[Steve/Gerhard] Typically a prospective camera buyer understands what information they want to extract from the machine vision system.  For example, the smallest size detail they want to see, how fast the machine vision process needs to take to maintain the production line speed, the environmental aspects and how it will influence the machine vision system, and how the formulated decisions need to be transmitted and ultimately used by the plant.  The experienced imaging machine vision specialist will then have a fairly good ideal of the correct camera type and more important the correct machine vision system.

[Jin/Dave] The more that a customer can tell us about the application and what they want to accomplish, the better we can determine what camera features will be most important to them and can then assist them in choosing the best camera for them. Insight that would be helpful includes:

• Purpose of usage
• Type of application, size of application, and the process of application
• Buyer’s requirement and specification

[Joachim] We need information on resolution, frame rate, optical and electrical interface in order to deliver the proper camera.  If the customer does not have this information, then a detailed description of his application is needed. This contains the space, lighting, object, and the required content of the image on which the software will operate. 

[Christophe] As a camera supplier, ATMEL expects to know the application, the mode of implementation, and if possible (if the client is able to qualify his need) a list of the expected features, with a ranking like: mandatory/expected/preferred/optional. As complementary information, it is interesting to get an idea of the target price (of course linked to a realistic and factual business potential) and the context of the project (i.e., R&D prospective work, upgrade of an existing system or second source).

[Dave G.] That they explain their application to us in enough detail so that we can define the critical performance requirements.  Often, the user does not even know such requirements exist.  Most of the weakness comes from very limited understanding of how images sensors work.

6. Are there specific application issues that must be understood insofar as selecting the correct camera for the application is concerned?

[John] Always.  As in all vision applications, optical, mechanical and machine control issues are central to problem solution.  Also, the customer’s choice of frame grabber, processor and other system components all become key in a properly operating vision system.

[Ilias] Please see answer above.

[Keith] This is the same answer as number 5. 

[Marty] There are a range of issues that should be considered when selecting a camera for machine vision applications.  Camera performance requirements, optics, application geometry, lighting, software issues, and computer performance issues are all important issues that must be understood when selecting a camera. There are many issues that may not appear important, but which should be reviewed when selecting a machine vision camera.  An example of this is that people often request color cameras without realizing the downside of that choice.  Monochrome is much easier to handle in most cases offering improved sensitivity, improved speed, reduced computational requirements, and higher effective resolution.  In many cases, monochrome cameras will achieve a more satisfactory result than color and with fewer integration problems.

[Bernhard] - Application task, project aim:

• Image Quality
• Resolution, speed, interface
• Surroundings of camera (temperature, light, dust etc.)

[Gunnar] It is important to understand the basics of image sensors, and know about their limitations that can cause unwanted artifacts, such as smear, blooming or readout-shifts (e.g. interlace scan or rolling shutter effects). It is equally important not to confuse the dynamic range specification of an image sensor with the actual S/N ratio of the camera. The electronics of the camera, meaning the CDS and A/D circuits and power supply noise (DC/DC converter), will always govern the actual S/N ratio at the output. Furthermore, with clock rates in the order of 40 MHz, and on the increase, noise sources need to be clearly understood.

[Loren] The ones that I have listed on Item #5 can apply here as well.   If the user understands all of those items, they have a good understanding of their application.  

[Bill] Movement of the object, speed, size, and degree of detail required.  These attributes allow us to determine the required resolution, frame rate, data rate, and sensitivity characteristics required.  The issues covered in #5 are all application issues that must be understood.  In addition, connectivity formats and frame grabber specifics, as well as operational requirements such as triggering, need to be understood before choosing a camera.  

[David C.] Each application tends to be unique and each customer wants to differentiate themselves from their competitor. We want our customers to succeed, so it’s helpful when we have customers that are willing to share their specific application issues with us. At DALSA, we believe in working closely and very confidentially with our customers to find the optimal solution for their application.

[Steve/Gerhard] Illumination and imaging optics are important factors, and the available time to do the image recording and subsequent processing typically determine the requirements.  The application and the imaging focus are important factors; size and shape of holes in mechanical parts are very different from texture analysis or the quality measurement of color LCD displays.  Camera characteristics for life science or biotechnology applications are more stable and low noise oriented.  Therefore purely the application defines the requirements.

[Jin/Dave] The better understanding of the application that we have the more likely we are to design a camera that the customer will be happy with. Insight should be shared on:

• Software requirements
• Interface requirement
• Output requirement
• Measurement requirement

[Joachim] This cannot be answered generally. All applications have to be treated individually in order to achieve the optimum result. Please also refer to 5.

[Christophe] Yes there are. For instance, applications using extreme wavelengths (UV or IR) require choosing the camera carefully.

[Dave G.] Timing seems to cause the most problems.  Stray light is another.  Usually the big items such as ‘‘do we need color?’‘ are understood but the factors that point to which color imaging technique would work best are understood poorly.

7. How important are camera properties such as resolution and signal-to-noise in machine vision applications?

[John] Depends on the application.  Sometimes yes and sometimes no.  Often component price overrides image quality issues.  A proper balance for each application is key.

[Ilias] The importance or weight of each of these properties depends on the individual application. For example, resolution is critical for large PCB inspection and other applications where a large object has to fit in one field-of-view. On the other hand, signal-to-noise ratio is very important for applications where subtle contrast differences have to be measured or image acquisition is taking place under low light environments.

[Keith] Resolution and the S/N ratio are extremely important for machine vision applications (refer back to number 5).  As minimum defect sizes decrease, high-resolution cameras are needed with better S/N ratios.

[Marty] Some people request high-resolution without realizing some of the trade-offs that they are making.  The camera buyer should choose the lowest resolution that will do the job so that data processing loads and product costs are minimized.  Higher resolution generally means lower speeds and higher costs but often without any compensating benefit.  More pixels means more analysis – this means that you may require a higher performance computer (or may even mean that you cannot achieve the throughput that you require).  Many people do not realize that their choice of lens and the accuracy of the focus setting can easy affect the true resolution of the imaging system.  A high-resolution camera needs a high-resolution lens that is properly focused.

Certified System Integrator Program

Set Yourself at the Forefront of the Global Vision Market

Vision system integrators certified by A3 are acknowledged globally throughout the industry as an elite group of accomplished, highly skilled and trusted professionals. You’ll be able to leverage your certification to enhance your competitiveness and expand your opportunities.

GET CERTIFIED

[Gunnar] Resolution is directly proportional to either the field-of-view or smallest feature to be seen by the application. Resolution comes at a price, though. It is important to calculate the requirements carefully, to avoid overkill. Many basic applications are still solved by VGA resolution cameras. Greatly improved sub-pixel interpolation algorithms can provide 1/10 of a pixel accuracy, providing cost effective solutions using simpler cameras.

As discussed under question 4, signal-to-noise ratio needs to be carefully considered, especially for high-end metrology applications.

[Loren]  Machine vision applications cover a wide gamut.  In general these properties are growing more important everyday.  Inspection requirements are more demanding.  Areas to be inspected are larger and the resolvable features smaller.  E.g., Intel Pentium processors have increased surface area to inspect and the pin grid array is more densely packed.  The result:  higher resolution requirement.  The density requires a better quality image to resolve the finer details.  Signal-to-noise ratio plays an important part in providing a high quality image to process.

[Bill] Resolution is important once the choice is made to leave analog EIA standards, which we are finding more and more as camera abilities and PC processing improves.  Improved resolution and processing power are allowing us to address new applications that were once considered too difficult or expensive to pursue.  Signal-to-noise for most machine vision applications has lessoned in importance as cameras have improved in this area.  However, signal-to-noise is still important for more scientific or medical applications, but less so in standard ‘industrial’ applications.  S/N generally becomes more of an issue when fine gray scale measurements are required, or where we are approaching the threshold of the detail or resolution capabilities of the camera.

[David C.] These features are critical in machine vision applications; this is why DALSA offers the highest resolutions in line scan technology with up to 12K pixels and a range of 1 to 22 megapixels in our area scan technology.  With a refined noise reduction process, DALSA‘s foundry enables sensor technology with the lowest possible noise, >72dB at room temperature, without the need for cooling.  

[Steve/Gerhard] In the majority of machine vision applications these properties are not that critical, since generally speaking the information users want to extract from the application images are not that demanding.  However, there are some very demanding applications where these properties are very important as the higher resolution and better signal-to-noise performance of the camera allow for extracting details the user is looking for, not available with lower quality cameras.

As mentioned earlier with the bottle inspection example, high dynamic range in the camera can give the final advantage.  High throughput applications in pharmaceutical technology or biotechnology have demands that are more focused on long-term stability and signal-to-noise ratio of the camera systems.  Looking at quality control applications with microscopes where various images have to be recorded, for example in wafer control, higher resolution cameras can yield to less images and faster control.

[Jin/Dave] The application will determine how big a factor these features will be.  For example if you are trying to identify small features between similar parts, a higher resolution would be needed than just if you are trying to detect the presence of a part. For machine vision applications, noise can be a serious issue. Our specifications tend to be conservative. Therefore, giving the customer better results than expected.  Other parameters:

Bandwidth and resolution; Determined by the general formula of 80 lines per Megahertz, it immediately and directly relates bandwidth to resolution. For example, a 480 TVLH (?80) color video signal requires 6 Mhz bandwidth without loss to pass it through the system. Any loss in bandwidth in the system results in a direct resolution loss. Bandwidth loss is cumulative.

Signal-to-noise; Noise is measured as a ratio of full amplitude, 100 IRE signal to RMS (root-mean-square) noise. The higher the ratio, the better the camera signal. S:N is not directly accumulative within the system. S:N for the system is equal to or less than the lowest S:N device in the system. Noise generally appears as ‘‘snow’‘ in a video image. Tune a TV set to an unused channel and you can view noise.

[Joachim] Resolution and frame rate are usually the first properties for a camera selection. Signal-to-noise ratio, dynamic range, etc. are technical properties that are essential for the quality.

[Christophe] Choosing the right camera involves making the following choices: area scan or linescan camera, analog or digital output, pixel number, size, ... also light wavelength, image rate (and thus integration time). Then, once the type of camera is defined, we can come to a second level of discussion where advanced parameters are to be considered: resolution, signal to noise ratio, sensitivity...(one depending on the other) Definitely, resolution and signal-to-noise are key parameters, which roots are at the sensor level. So, I would add: ‘‘trust first the camera suppliers who have sensor design (and ideally manufacturing) in-house capabilities!’‘

[Dave G.] They both have to be sufficient to allow needed operating margin in the application.  They mean nothing in the absolute.

8. What do you see is the trend for camera connectivity – FireWire, USB, Camera Link®, and Gigabit Ethernet (GigE Vision™) – and why do you suggest that?

[John] Gig E - it's ubiquitous and inexpensive.  Also, 10 Gig is just around the corner.  FireWire is being proliferated less and is limited to 30-foot runs.  USB is, for the most part, a consumer/laboratory bus and not an industrial one.  Camera Link currently outperforms all other connectivity options and will continue to be a significant player for that reason.

[Ilias] Sony has been a long proponent of 1394 interface due to features like ease of use, low cost connectivity, good standard definition and robust image transfer. Our commitment to 1394 will continue with the introduction of 1394.b (800 Mb/s) products in the very near future. Recently, we also introduced a Camera Link compatible family of products targeting customers with a requirement for a point-to-point interface and absolutely deterministic image transfer. The Camera Link compatible line will enable our customers to smoothly make the transition from analog to digital technology. Finally, we always keep a close eye on all other potential standards and consider their implementation for future product designs.

[Keith] As camera resolutions becomes larger and frame rates faster, the demand for faster interfaces are required.  The obvious choices are Gigabit Ethernet and Camera Link.  FireWire (A/B) and USB 2.0 are not fast enough for high-end applications – especially as bit depths increase to 12-bits per pixel.

[Marty] Prosilica believes that FireWire represents the most cost effective and easily implemented of the currently available interface standards.  Apart from the obvious cost savings associated with firewire (low cost, widely available standardized cabling, low cost computer interface hardware, etc.) the standardized software interface known as DCAM (IIDC) makes it very easy to integrate FireWire cameras into machine vision applications. 

FireWire will continue to have a growing place within machine vision, but Prosilica believes that there are certain advantages to Gigabit Ethernet that will eventually impact the market.  Once the GigE Vision camera interface standard has been finalized and proven, Gigabit Ethernet will offer FireWire’s ease of integration together with increased speeds and cable lengths.  The word ‘‘Ethernet’‘ implies an ease of use and low cost that, while currently untrue of Gigabit Ethernet, will drive the market demand for cameras based on GigE.  In time, the ease of use, high performance, and low cost of Gigabit Ethernet will make it the interface option of choice. 

Camera Link will remain for some time for very high data rate cameras due to its parallel structure, but it is too expensive and complicated to remain as one of the main interface options for the wider machine vision market. Camera Link will be displaced in most applications by Firewire and Gigabit Ethernet. We do not see USB as a viable interface choice for machine vision, although it may find application in various desk-top imaging applications such as microscopy.

[Gunnar] In many markets we see a clear trend towards serial interfaces, with the objective of eliminating the frame grabber. This is most evident in Europe and parts of Asia. Japan is just in the process of migrating to Camera Link, where analog interfaces still make up the majority of the systems. Serial interfaces are, therefore, not being discussed much in Japan. The US market is also beginning to adopt serial interfaces, but not at the same pace as in Europe. Camera Link will not be completely displaced in the future, but will be reserved for high-end multi-tap cameras where the serial interfaces do not provide sufficient bandwidth. GigE is positioned to become a widely accepted interface standard, as it is based on well-established technology with a clear road map for the future (10GigE).

[Loren] Oh to have a crystal ball!  Due to the nature of the interfaces, we will probably see USB and FireWire capture a majority of the lower end market.  GigE will find its own niche where long distances and relatively low data rates abound.  Camera Link will hold the higher end of the market.  With the demand for yet higher speeds and longer distances, I expect to see interfaces such as fiber channel and infiniband also start to show up.  They can be implemented over copper or fiber optic.

[Bill] All formats offer specific advantages and limitations.  FireWire has been gaining in popularity, but there are still limits to frame rate that often limit its use in robust applications.  Camera Link can fill this void, and with more frame grabber manufacturers offering lower cost boards with standardized formats, popularity is also gaining.  Inquiries into USB and GigE have been increasing, but we are still not finding these formats requested as often as FireWire or Camera Link for what we consider to be true machine vision applications.

[David C.] The need for greater bandwidth to support cameras is driving the trend towards Camera Link. The need for a remote controlling system is driving the trend toward Gigabit Ethernet. The high speeds of DALSA’s cameras require high-bandwidth, which is why Camera Link is used extensively in our product line. In addition to its networking capabilities, Gigabit Ethernet provides a good solution for sending camera output over distances ranging to 100 meters. Firewire and USB are well recognized in the industry, but do not play a role in the markets we serve because they simply do not have enough bandwidth to support the speeds of our cameras.

[Steve/Gerhard] Camera connectivity in general should not be a critical issue to users of machine vision systems, in that what is most important is the overall performance of the machine vision system, and not the individual connector type.  However, there are advantages for faster and multiplexing communications between different locations within a plant where machine vision is being used, so in general faster camera connectivity would be better.

The lower cost Gigabit Ethernet might become more important, if the definition of the standard is successful and enough companies support it.  Nevertheless it will also depend on the balance between pre-processing in the cameras and central processing in a dedicated computer with multiple cameras.

[Jin/Dave] We believe the trend is toward simple serial type communications.  In applications that are short distances from the processor connections such as FireWire or USB will be the choice.  In systems that are spread over a larger area GigE will be the interconnection of choice.

The current trend is to use FireWire and Camera Link due to the following benefits:

FireWire:

• Great use with multimedia peripherals such as digital video camera and other high-speed devices like the latest hard disk drivers and printers.
• It is integrated into Power Macs, iMac, eMacs, PowerBooks, iBooks, and the iPod.
• Operates up to 400 megabits per seconds (1394a) and operates up to 800 megabits per seconds (1394b).
• It is cross-platform implementation of the high-speed serial data bus.
• Simplified cabling, hot swapping
• Enables easy connection of digital consumer products – directly to a personal computer

Camera Link

• Easy camera/frame grabber integration, connectors, cables, data format, and control signals are all standardized
• Industry standard cables mean you can buy in volume and get competitive pricing
• Camera Link chip sets have high data rates, perfect for today’s image data transmission requirements and expandable for future needs.

In our opinion, future applications are focused at reducing cost. The need for frame grabber is being diminished.

[Joachim] Camera Link will be the standard interface for high-end applications where huge data rates (see answer 3) are required. If the data rate is not a critical aspect, Camera Link will be replaced by frame grabber-less interfaces like Gigabit Ethernet or FireWire. When comparing USB versus FireWire, IEEE 1394 is a more mature and flexible interface technology solution for industrial and professional applications. IEEE 1394 also provides the opportunity to increase performance and bandwidth in the future and this makes it flexible enough to be the right answer for upcoming applications.

To be successful in the marketplace, all new and existing interface technologies must meet the requirements: an easy to use connector concept and easy to use camera configuration, that includes plug and play, an industrially proven design, and a widely accepted, logically standard interface.

[Christophe] As for us, at ATMEL, we believe all interfaces will cohabitate in the future, depending on the application. The move from analog to digital is really engaged and benefits to this ‘‘low bandwidth’‘ interfaces USB, IEEE 1394, Gig