Machine Vision-Based Inspection for Electronic Assembly. . . .by Nello Zuech, Contributing Editor

Application-specific machine vision-based systems addressing applications in electronic assembly have been around for a long time, almost as long as the machine vision industry itself. Early on, the requirement was to verify the presence of a lead in the appropriate hole. This was not widely accepted because the board populating equipment often came with sensors able to detect if the lead did not go into a hole. Significantly, it did not detect if it went into the correct hole.

About the same time companies responded to the requirement for solder joint inspection by offering systems capable of assuring the solder was present and not shorting. This, however, was only a partial response. Primitive 3D techniques emerged capable of verifying wetting angle of the joint and surface conditions but the industry did not embrace these systems. High false rejects were experienced and, in the case of solder joint evaluation, experts debated whether the visual conditions really reflected a good joint. Furthermore, all these systems were very expensive.

Along the way the electronic assembly industry embraced surface mount designs. While intuitively some of the early application-specific machine vision systems targeting board assembly applications might be suitable, over time it has been determined that the systems had to be adapted with new features to improve reliability and reduce false rejects. The surface mount line offers a number of opportunities for vision systems: post solder paste verification and inspection; post component placement and before reflow presence/absence verification, correct component verification, placement accuracy; post solder reflow inspection to assure correctness of placement and presence/adequacy of solder and solder joint itself and no shorts caused by soldering; and post-wave to verify solder joint integrity especially on lead-through-hole-based components. In the case of post-solder applications both optical and x-ray-based vision systems are now available.

Both 2D and 3D-based vision systems have emerged in response to the requirements of the electronic assembly market. In the case of screen-printing, 3D systems offer the ability to inspect the solder paste both for placement accuracy and volumetric properties. While most component placement systems are 2D, some are 3D. Both visual and x-ray-based systems are available for post-solder inspection that perform 3D inspection. In the case of visual systems, the 3D surface properties are inspected and in the case of x-ray, the 3D internal properties of the solder joint are inspected.

Today there are at least two drivers that require all serious assembled board manufacturers to deploy these application-specific machine vision systems. The components themselves and the interconnects are becoming smaller making placement accuracies to obtain a good solder joint ever more demanding. And the response to RoHS international standards eliminating lead from the solder has resulted in changes to the viscosity of the solder itself presenting challenges to wetting properties and corresponding placement of the solder with respect to the component interconnect.

Given the importance of yield management, machine vision is finding more applications. 3D systems exist to monitor board flatness, which is ever more critical given the smaller interconnects and corresponding pads. Systems are also now available to inspect the screens used in screen-printing to assure pattern correctness and completeness.

Significantly, the underlying technology that is the basis of both 2D and 3D machine vision is becoming ever more relevant for applications in the electronic assembly industry as well as more powerful. Cameras now offer greater and greater resolution, speeds, and better color properties. Lighting is more capable with the developments associated with LEDs making multi-directional and sequential lighting more cost effective. Needless to say the underlying microprocessor and compute technology makes it possible to perform more compute intensive algorithms leading to more rigorous performance.

The result is that the capabilities now available include precision metrology, color-based recognition, optical character recognition, better characterization of surface conditions and CAD file compatibility. The result is better detection with fewer false rejects and systems easier to train on new board designs and interface to rework stations.

With all this in mind we canvassed input from many of the suppliers of application-specific machine vision systems addressing assembled board applications in the electronic industry.

The following provided the responses to our questions:

• Stacy Johnson, Product Marketing, Agilent Technology
• Jim Gibson, President, Landrex Technologies, Inc.
• Shavi Spinzi, Director of Marketing, Orbotech

1. What are some specific applications in the electronic industry that your company addresses with machine vision technology?
[Stacy Johnson – Agilent] - We offer a broad range of inspection solutions for a broad range of applications in the electronics industry.  Our optical inspection portfolio supplies 3D solder paste inspection systems, pre and post reflow component inspection systems and 3D X-ray for post-reflow solder inspection.   Our systems are utilized in many different electronic sectors including mobile phones, servers, automotive, etc.

[Jim Gibson - Landrex Technologies] We are a global test solution provider for the printed circuit board industry. Landrex’s offerings in the electronic industry that are addressed with machine vision technology are the Optima series of pre- and post-reflow inspection systems. The Optima™ 7300 series is targeted at post-reflow inspection of all aspects of a printed circuit board including parts, solder joints, board defects, and pin-tips. This machine family has two offerings, the Optima II 7301 Express™, which is a low cost, high performance system and the Optima II 7310 Extreme™, which is an extremely high throughput version of Express. On the in-process side of the oven, Landrex offers The Optima™ 7210 Optical Process Test system (OPT). This system performs post-place and 2D post-paste inspection and measurement and provides statistical process control information.

[Shavi Spinzi - Orbotech] We offer inspection solutions for all stages of the SMT production line: ultra fast combined 3D/2D for paste volumetric measurement, 3D for placement inspection, post reflow and wave inspection of solder joints. Our products are designed to cover the demands across the whole assembly inspection spectrum from high-speed mass production of gaming, computer and portable devices to highly flexible production in the mid-sized EMS segment.

2. Which of your applications uses 3D techniques as well as 2D techniques? And why?
[Jim] The Optima II 73xx series uses a very flexible combination of lights and cameras to assess some characteristics about the 3D profile of solder joints.  This line of machines has five cameras, four angled and one center, as well as a “dome” of LED lights that can be configured in any number of ways. The combination of angles of cameras and angles of lighting directions allow the 73xx series to assess the reflectance properties of both flat and non-flat surfaces. In particular, the system is able to check solder joints to see if they have certain 3D properties such as being convex, concave or flat. At present, the 73xx series does not reconstruct the height of points along the solder joint. However, the generic composition of the hardware allows for future software development that would not only assess the 3D properties, but also to reconstruct them. At some point in the future, the system may exploit this ability.

[Shavi] For solder paste inspection, Orbotech offers the Symbion P36 with a combinational 3D/2D sensor for accurate volumetric measurement of solder paste. It is widely accepted that height and volume information is a critical parameter in the paste process that has a strong influence on end-of-line yields. In this mode, 3D actually complements the basic area data that 2D provides to give excellent test coverage and the user gives the most comprehensive view of the process. For component placement a 2D approach to catch presence/absence and the position of components is sufficient. For solder joint inspection, Orbotech offers a 3D system that employs a high-resolution top camera with four angled cameras. Combined with multiple angled white lighting, we are able to accurately characterize the profile of solder joints, and of course get visual access to otherwise concealed or partially concealed solder joints such as those on J-legs ICs or increasingly high-density edge connectors.

[Stacy] Agilent's solder paste inspection and X-ray system both utilize 3D inspection techniques.  For solder paste inspection having the 3D volume and height information is important because volume has been linked to long-term joint reliability.  If you only measured 2D you would not have that critical information.  A main driver for X-ray is hidden joints and the 3D is needed because of the double-sided boards that our customers build.

3. What are critical machine vision system performance criteria for each of the applications that you address?
[Shavi] The following are vital:

1) Irresepctive of the inspection stage, all machines have to meet SMT line speed with 100% board inspection (not sampling).
2) In paste inspection applications accuracy and repeatability of measurement is critical with Gage R&R <10%
3) Low false alarms - FPY >90% for post reflow and >95% for paste/placement
4) Detection level >95% - this means see all defects including lifted legs in the post reflow stage
5) Fast and easy set-up.

[Stacy] In AOI (automated optical inspection) customers often utilize accuracy and repeatability to determine if an AOI system will meet their measurement needs.  For AXI (automated X-ray inspection - in our case it's 3D) low false call rates is a main driver for adoption and implementation.

[Jim] For all machine vision applications there are five main criteria – false fails, false accepts, programming time, speed, and of course, cost. Most customers consider speed and cost parameters in their preliminary rankings of the products on the market. However, false fails, false accepts and programming time can have just as big an impact on the customer’s economic bottom line as speed and system price.

The Landrex Optima series machines for pre- and post-reflow optical aim to have exceptionally low false fails and false accepts in the range of 20-50 ppm which remain stable over long periods of use of the systems in line at real customer sites. The Optima series machines also have exceptionally high speeds, especially the Optima II 7310 Extreme for post-reflow analysis. Each type of machine has a programming time of 2-4 hours. Finally, each machine starts at an asking price below 100K.

The Optima 7210 OPT system has additional performance criteria. Because it is a measurement system as well as a defect screener, it adheres to very rigorous accuracy and repeatability specifications. In general, a measurement system must be 10x more accurate and repeatable than the machines that manufacture the boards, in this case the pick and place system, in order to provide useful measurement information about the process. For an 0402, for instance, the system must have an accuracy of +- 100 microns and a repeatability of +- 10 microns. For an 0201, the requirements are even more stringent. In general, this type of requirement is known as a GR&R of 10%.

4. What changes have been taking place in the technologies that are the basis of the machine vision systems used in the electronic industry that has resulted in improved performance?
[Stacy] In my opinion, the most noteworthy would be the evolution in camera technologies, which are assisting with the continuous improvement of the AOI product inspection speed and accuracy.

[Jim] In the past 5-8 years we have seen dramatic improvements in the hardware components of machine vision systems. For twenty years in the machine vision field, engineers suffered with extremely limited camera and lighting choices and also relatively slow computers with limited memory. Machines constructed of these hardware pieces could only use the most rudimentary image processing techniques to solve their vision problem. Since 2000, the hardware area has exploded with newer, faster, cheaper, more reliable, bigger array cameras. In addition, color cameras are now an option with all the same parameters as the gray scale versions. Computer speeds and memory capacities have increased exponentially, while prices have decreased at shocking rates. Finally, the most dramatic advance has been in the lighting industry. With the advent of LED technology, especially white LEDs, it is now practical from a cost and speed perspective to provide white strobable light in many different shape configurations. All of these hardware advances give the machine vision the capability to “see better” and the basic computing platform to process information in ways we couldn’t have previously imagined.

[Shavi] An AOI system is a multidisciplinary machine comprising optics, hardware and software. Advancements in all of these technologies enable us to create products to meet the growing demands placed on inspection by the electronics industry. Cameras with faster frame rates that ensure high image quality coupled with high-performance frame grabber hardware enable amazing data read-out rates and a subsequent speed of operation that was traditionally the bottleneck especially for 3D AOI. Advancements in lighting technology - the cornerstone of image processing – such as LEDs, are also giving huge benefits in terms of intensity and the ability to control the lighting environment at camera frame rates up to 200 FPS. Finally, new software platforms such as JAVA have had a profound effect on how we can run routines on hardware and dramatically reduce time-to-market for new developments.
5. Where do you see breakthroughs coming in the specific technologies that are the basis of machine vision systems used in the electronic industry that will result in further improvements in the near future – next three years?
[Jim] The next wave of breakthroughs will be in the advances in brains of these systems, namely provided by the computer vision algorithms. New algorithms will allow us to do existing tasks easier and more reliably and to do new tasks that were previously unimaginable. For instance, thanks to computer vision algorithms we are now able to offer variable data about part position rather than just “go/no-go” defect analysis. This variable data will truly revolutionize the field of process control for PCB production. The hardware revolution from the past five years will lead to an unimpeded software revolution in the field in the next five years.

Certified System Integrator Program

Set Yourself at the Forefront of the Global Vision Market

Vision system integrators certified by A3 are acknowledged globally throughout the industry as an elite group of accomplished, highly skilled and trusted professionals. You’ll be able to leverage your certification to enhance your competitiveness and expand your opportunities.

GET CERTIFIED

[Shavi] Multiple core processors will create a breakthrough for data processing and give impetus to new levels of machine performance where tasks that are typically executed sequentially can be performed in parallel. State-of-the-art classifiers for true multi-dimensional planar separation of interrelated characteristics will drive a new level of defect detection with near to zero false calls.

[Stacy] Same as above.  I believe the evolution of camera technologies will drive further improvement in AOI in the next 3 years. 

6. Are there market changes in the electronic industry that are driving the adoption of machine vision? E.g.. What impact does RoHS compliance have? What about new IC packages and interconnect densities?
[Shavi] AOI is experiencing a steady and healthy growth rate in the area of 7 – 10%. The introduction of RoHS has no doubt influenced the decision to introduce AOI but is not a major deciding influence. Increasing package miniaturization and legislative pressure on companies, however, is. The bottom line with miniaturization is that the human eye cannot cope and the industry is resorting evermore to the reliable alternative that machine vision poses to complement or replace both manual visual inspection (MVI) and electrical test. Legislation – which includes RoHS – is putting pressure on manufacturers to become more transparent and provide more comprehensive documentation on the whole process. In this area, AOI offers major benefits in terms of both the visual and quality data that it provides on the SMT process.

[Stacy] For AOI and AXI the ever shrinking dimensions of packages/devices on SMT boards is driving adoption.  Additionally for AXI, access is an issue.  The prevalence and increased usage of BGA and CSP (ball grid array and chip scale package) packages that inherently have hidden joints are driving AXI needs.  I believe that lead-free is a driver in that if you use machine vision (AOI and/or AXI) you can bring your processes to stability faster.

[Jim] The truth is that it is harder and harder to test printed circuit boards. With device pitches getting smaller, boards denser and more complex, and materials changing from lead to lead-free, a strategy of relying solely on human inspection, ICT, and functional test will result in lower yields. Additionally, the margins most companies get per board are decreasing. As a result, most companies see a compelling reason to do 100% optical inspection to increase yields, decrease manufacturing cost, increase quality, and in the long run grow market share.

7. What impact do these changes have on the inspection requirements for machine vision systems? And how will machine vision systems have to change to address these more demanding requirements?
[Stacy] I do not believe lead-free material set is driving the change in requirements for machine vision but the drive towards smaller devices is.  The requirements for accuracy and repeatability are continuing to get more challenging as the devices get smaller.

[Jim] Optical inspection systems have to evolve to meet these demands. The hardware and software has to see tiny parts, understand variations in the new materials, have enough horsepower to meet the speed of the line, and be reliable enough to run 24/7 for long durations without maintenance. The machines have to deliver minimal false calls and false accepts without a lot of human intervention. A false accept can mean that the board may fail in the field. A false reject results in expensive, labor-intensive human validation of the machine’s output. False flags and false escapes can have a very big impact on the economic returns for many companies. The bottom line is that the machines have to be able to adapt on their own to changing visual aspects of the products being built. While this sounds simple and logical, it is quite difficult to engineer a system that can both adapt automatically to new conditions and provide reliable inspection results given the existing conditions.

[Shavi] There is paradox here as machines must become more complex and packed with more power to deal with these demands. The user – on the other hand – has neither the time nor the image processing know-how to harness and exploit this power. Inspection machines, therefore, will continue to offer a high degree of intelligence but the focus is to make it easily accessible and controllable for operators of varying skill levels. As well as the power to process and manage data, AOI will adopt a predictive approach to inspection by having built in intelligence that is able to both characterize and predict process variance. We are entering the era of unattended AOI.

8. As a supplier of machine vision systems for the electronic industry what are some challenges you face in marketing machine vision systems?
[Jim] The biggest challenge that we face is getting the customer to understand the differentiation between our products and other competing products in the market that offer a continuum of capabilities. The truth is that anyone with a camera and a flashlight can cobble together an AOI system. Every year we see a few 'home-grown' AOI systems put on the market, and we also see people actually buying them because of their usually low price tag. Surprisingly, one can get a limited base level of performance with almost any AOI system. However, to get to world-class levels of performance, with error rates in the range of 20-50 ppm, that stay stable from a small run to high volumes and across different types of boards, one has to do a significant amount of engineering. Making a machine that automatically understands what it is looking at 99.99% of the time is a very hard task. We as humans perform visual recognition tasks all the time in different situations. However, we have had millions of years of evolution on our eyes and more importantly our brains.

During a short evaluation at a customer site, it is difficult for the customer to understand the difference in technologies that will give them excellent performance in both the short and the long run because most machines fair well in the short run and the long run is not easily demonstrable. Our products have 20 years of engineering, science and practical experience behind them. We test them extensively in line at customer sites on real boards. We want to see our excellent performance last well beyond the evaluation to running 24/7 on millions of millions of parts in real customer use environments. We provide this customer data as part of our sales literature because it is important for people to understand that performance in a demonstration is very different from performance over the long haul.

[Shavi] Orbotech’s focus is on innovation and adoption of new technology for both the inspection requirements of today and the challenges of the future. There are many systems on the market and most are set up to demonstrate good initial performance and tick all the feature boxes. The issue for the market is to distinguish between technology that has limitations and will create a bottleneck in the future and systems built on new technology that are well equipped to adapt to their needs. In our marketing approach, we focus, therefore, on the four basic elements of AOI, namely: ease-of-use, detection, speed and accuracy.

[Stacy] One large challenge is awareness.  Even though Agilent has a portfolio solution it is interesting that many people do not know we sell 3D solder paste inspection systems, for example.

9. What are your thoughts on the future of machine vision in the electronic industry?
[Shavi] Rather than a necessary evil, I believe that machine vision will establish itself and enjoy higher penetration into lines and processes and almost become a commodity in electronics manufacturing. The biggest challenge facing our industry is how to deal with the transition of electronic functions onto the silicon substrate. The result is less visible components on the assembly and although X-ray may be the obvious answer unless there is a breakthrough in sensor technology X-ray will remain a cumbersome, restricted and more expensive alternative to AOI. With cost continuing to be the biggest driver a more effective solution is to use machine vision to control the pre solder process. This will include intelligently combing data from different machine sources in order to better understand process behavior and predict non-conforming states.

[Stacy] With the adoption of lead-free and the advent of new and smaller packages we will soon see closed loop being more prevalent for pick and place and AOI and for the screen printers and paste inspection.  In the future, machine vision applications such as AOI and AXI will be ‘standard’ in production lines.  The increasing abilities of AOI and AXI to influence process control through defect prevention and defect containment is ensuring that machine vision will soon be considered a value add and must have rather than an ‘expensive test system’.

[Jim] There are four trends happening in the electronics industry. The first is that we are able to build more complicated sophisticated machines. The second is that the asking price for these machines, almost irrespective of fields, is dropping rapidly. The third, which is almost a consequence of the second, is that factories are constantly on the move to find the cheapest labor source. Finally, in order to keep up with rapidly changing products and to keep the factories at full capacity, each factory is making a larger mix of products. As a result, we see that inexperienced factories are making more complicated and mixed lines of products for dramatically low prices. A scenario like this one demands automation at every level, including optical inspection of the electronics. Thus, we will see much less reliance on human intervention. In addition, not only will the machine vision systems being used for defect inspection, they will also be used in an integrated way to provide information about the process in order to help the factories build product right the first time.

10. What advice would you give to a company investigating the purchase of a machine vision system for an electronic industry application?
[Jim] Surprisingly, personnel costs are really the hidden costs of AOI machines. The big issue is that people do not realize that the visual appearance of printed circuit boards, pasted pads, components, and solder joints can change dramatically across boards and even within one board. An optical inspection system must be able to handle this naturally occurring, acceptable variation and to distinguish it from real defects reliably and robustly.

Most AOI systems ask the programmer to show it all the types of variations it can expect to see, and to give it a recipe for how to inspect for defects given these types of variations. If a new variation occurs, the programmer must “dial in” this new occurrence. If we assume that new variations happen all the time, most machines require almost constant attention and a highly skilled programmer to keep them at acceptable false fail and false accept rates. If a programmer is not available to fix the issues, the companies pay for the problem in having humans verify the false calls. Most people don’t realize the staffing commitment they’ll need after they purchase the machine. This level of staffing can increase the cost of an AOI system by 100%, which is way too high.

If we had one piece of advice to offer customers, it is to evaluate how well an AOI machine can automatically handle acceptable variation on PCBs without the aid of the human programmer.

[Shavi] Companies often make the mistake of focusing on the test aspects that will solve their immediate problem with little consideration for the future or the changing environment. When we talk to customers, it is not uncommon to find a previous AOI purchase standing unused in the corner because it eventually failed to meet expectation. Consider if the platform has the potential to adapt to meet new challenges such as new component formats or board designs. Give thought to your employee turnover and skills. It may seem to be an advantage to drill down to individual component algorithms but can that knowledge be passed on? Lastly, if your enterprise is on course for expansion will your machine meet the growing demand. Often, the simple fact that the machine cannot be connected to a central shop floor data collection system can seriously diminish its effectiveness as an asset.

[Stacy] Look for leverage.  An entire portfolio solution with the ability to help tailor the test strategy and implementation, like Agilent offers, will give you the best test plan for your production line.  Leveraging a full line solution from a company like Agilent also brings hidden cost savings from things like training because the portfolio systems operate similarly or the same in some cases. The lower infrastructure costs and the general overhead in addition to the data cross-talk among machine vision products is guaranteed to save manufacturers money.  Rather than engaging in long laborious evaluations with many suppliers to save a few $K on a point solution, why not invest in a portfolio solution where your cost savings can be customized and maximized and continuously evolving over time?