Teleservicing and Machine Vision Systems

Starting in the mid to late 1990s companies started offering some form of remote diagnostics and it was not long before some machine vision companies subscribed to this customer benefit. Growing a machine vision business invariably requires expanding a global presence. Supporting such a presence especially post-installation is most cost-effective using teleservicing tactics. The ability to perform diagnostics remotely can reduce the cost of onsite service calls substantially. From the customer’s perspective, teleservicing is cheaper when supporting out of warranty product on the shop floor. Software updates can also be sent via telephone lines.

Some issues, however, include pragmatic ones like availability of a phone line and/or Internet access for each machine vision system. Other concerns include data confidentiality when a system can be remotely accessed at any time. This concern can be mitigated by having the only time a machine can be accessed is if the machine operator himself activates the remote diagnostic tools to initiate the remote service. Once connected the remotely stationed service engineer can communicate with factory personnel via telephone or text messages viewed on the system display.

In a typical scenario once connected the service engineer would have access to the camera output and camera parameters, various machine parameters, status of I/O points, etc. Any changes initiated by the service engineer could then be downloaded. The popularity of cell phones, wireless modems and products like personal digital assistants (PDAs) is encouraging machine vision vendors to develop next generation remote diagnostic techniques. Some capabilities might include calling up a system’s complete history, including reasons for past problems and corrective actions taken.

As with other articles, the material for this article was solicited from companies known to be suppliers of machine vision systems of one type or another. A number of questions were developed and forwarded to representatives in these companies and what follows is their responses to those questions in a ‘‘round table’‘ discussion format. Those participating reflect various segments of the machine vision industry – general-purpose machine vision and application-specific machine vision systems for several industries.

Participants in this discussion included:

• George Blackwell – Cognex Corporation
• Sean Willis – Eisai
• Rod Larsen and Tim Reardon – Key Technology
• Werner Goeckel – Lasor Systronics
• Boris Mathiszik – Machine Vision Products
• Endre Toth – Vision Components
• Vic Wintriss – Wintriss Engineering

As a supplier of vision engine components (vision processors, embedded vision processors, frame grabbers, smart cameras, smart sensors) has this been a capability that you now offer as an integral capability in your products?
George Blackwell: ‘‘Though way ahead of its time, Cognex pioneered online vision development assistance and teleservice capability as far back as 1994. Today, our network-enabled products make it easy to uplink from the device level network to the control and enterprise networks, so it’s possible to connect to them remotely. While we have the capability to do remote set up, and diagnostics on these systems, most customers are more concerned with the security of their corporate networks.’‘

Rod Larsen and Tim Reardon: ‘‘The capability for such a remote connection is included in all of the machine vision equipment we sell.  The data line connections to enable the actual use of this capability remain the responsibility of our customers.’‘

Werner Goeckel: ‘‘Yes, we have offered this feature for a long time.  We run the same central software so that we can ‘‘ look’‘ at the customers line running in real time.  If the customer allows us into his network, we can make online changes as we are watching his line run.  We have done this with customers around the world.  We also upload the latest version of software automatically, if he wants it.  Our systems contain ‘‘ self help ‘‘ menus and instructions for the operators.  We also have 24-hour hotline answering if the customer wants this service.  We do charge for the latter.’‘

Boris Mathiszek: ‘‘Yes, we have powerful off-line programming capability that is used to provide web support to our customers. They can upload images from their system through our web site, and our application engineers assist with remote debug and optimization assistance. We are supporting 600 systems in the field, spread out over 30 countries, so this capability has been of tremendous value in our operations, and to our customers.’‘

Endre Toth: ‘‘In the latest smart camera models based on newer chip technology the presence of pre-emptive multitasking makes it possible to apply more advanced teleservicing functions. In these cameras the communication processes run in parallel to the mission critical inspection or measurement processes in such a way that it has absolutely no effect on the mission critical processes.

Sean Willis: ‘‘We have had requests for remote troubleshooting using a modem which works through the PLC in the machine, but have not activated this function on a live machine due to the security and control issues involved (in the pharmaceutical industry). ‘‘
If so, can you describe the specifics of your implementation of this capability and the reason why you chose that approach?

Larsen: ‘‘Key Technology utilizes Net Meeting through a modem-to-modem or Internet connection to replicate the equipment's User Interface on a screen in our offices or on a laptop of one of our remotely located Field Service Engineers.  The User Interface is the best source for diagnosing problems and for understanding the current set-up and operation of the equipment.  High-speed networks obviously allow for easier and faster remote monitoring of the GUI.’‘

Goeckel: ‘‘Through training and having a ‘‘go to person’‘ at the customer site, most problems are solved without physically going to the site.  This capability is what differentiates the solutions oriented companies from the hardware suppliers.  Integration knowledge, problem solving, and follow up service are important to a successful installation.’‘

Blackwell: ‘‘Generally, customers prefer to log onto our support network to take advantage of our searchable knowledge base, downloads, case submission, and other 24/7 services. Alternatively, customers may choose to work interactively with Cognex application engineers to solve their problems. In this instance, customers would typically capture the output image results from the Cognex vision system or sensor on any workstation with TCP/IP capability. Then the vision processor image and job can be sent via modem or e-mail to a Cognex application engineer’s PC. The Cognex application engineer then checks system set up, determines the best way to acquire the image for a particular job, optimizes parameters, and applies vision tools appropriately. Though it’s possible for us to do all of this via remote access, most customers prefer that we supply emailed or archived files for them to reload on their own.’‘

Toth: ‘‘The latest smart camera based machine vision systems provide on-the fly setup as standard feature. This means that the operator is allowed to change parameters for an inspection on the fly, furthermore define, add, delete or experiment with a new measurement tool on a newly defined ROI, while the inspection on active ROIs are running without any impact. Once a new ROI is fine tuned the operator is able to activate it into live running mode.’‘

Vic Wintriss: ‘‘It is very cost effective, in fact I can't imagine doing it any other way.  To help our customers set up their web inspection operations and to trouble shoot, we would have to send a person on-site.  This way, we can operate his equipment from our factory, simultaneously converse over the telephone and avoid costly, time-delayed trips to the customer site. We have VxWorks built into the camera.  VxWorks handles Ethernet communications.’‘

In your opinion, what makes this capability possible today?
Goeckel: ‘‘The fact that more people are knowledgeable about computers and networks makes this kind of analysis must easier than even five years ago.  Although the systems are more capable, they are simpler to operate.  The growth of networks and database information that is easily accessible has also helped in this capability.’‘

Toth: ‘‘Internet enabled smart cameras with built -in web server offer a natural tool for remote diagnostics and more. ’‘

Mathiszik: ‘‘Our ability to emulate inspections off-line, with the only difference between real-time, in-line inspection and off-line inspection being that in-line the images are served from the camera whereas off-line they are served from disk.’‘

Wintriss: ‘‘For us, having Ethernet communications from our cameras makes this possible.’‘

Blackwell: ‘‘High-speed modem technology, the Internet, and application sharing software that allows you to take control of another user’s PC.’‘ 

Willis: ‘‘Advanced vision systems and PLC speed and technology.’‘

Larsen and Reardon: ‘‘PCs as the communication center of our system, in combination with the communication capabilities inherent to PCs today. Data networking in the plant is becoming more common within the customer base we serve.  The ubiquitous and relatively cheap access to the internet makes connection around the world very practical.’‘

What does the user have to do to enable this capability? Does this mean he has to incur additional costs?
Larsen and Reardon: ’‘In the case of the simple modem, nothing. The system comes equipped with the modem. When he wants to use our service, he simply connects a phone line to the modem in his system, and we do the rest.  The user does need to provide for a connection to the equipment -- phone line or Ethernet.  In some instances, a router may be used to add security features and to connect multiple units of our machine vision equipment.  This would require additional hardware and installation costs. He must usually get approval from his IT/IS team, and set up the appropriate network addresses, etc.  But our standard offering includes a NIC or modem in our equipment.  So the only additional cost for the user is to run wiring and pay for the data connection, be it a simple phone line or a network connection.’‘

Wintriss: ‘‘We include this feature at no additional cost in every system.’‘

Mathiszik: ‘‘This is a standard support tool we offer all our customers without additional charge.’‘

Balckwell: ‘‘There is no standard implementation. Customers can use whatever method of service and support works best for them. However, our technical support department actively investigates new ways to improve our capabilities. For example, one scenario could be as simple as having the user log into an Internet portal or access point. There, a ‘‘firewall-friendly’‘ client installed on the customer’s PC would handle the communications logistics for a desktop streaming package. This would allow the applications engineer to take charge of a customer’s PC, do remote diagnostics, and walk them through the solution to their problem.’‘
Goeckel: ‘‘The user can get as much additional software capability as desired or needed.  If special software is written for his system we do ask him to pay for it.  The customer also pays for training and applications support.  Most customers do not use the total capability of the system.  If he wants to, we will teach him how to use now or sometime in future.  We do charge for this or he pays up front.’‘

Are users asking for this capability or enthusiastic when it is brought to their attention?
Mathiszik: ‘‘Yes, the reception is very good.’‘

Blackwell: ‘‘Very few are asking for this type of service. Most prefer that we email them files or archive files on our support network for them to reload into their own systems. In addition to direct service for our customers, we provide 24/7 online support and education. The support network also has a searchable knowledge base, downloads, and allows online case submissions.’‘ 

Willis: ‘‘Not so strongly requested at this point.’‘ 

Goeckel: ‘‘Some users ask, some don't.  It depends on what is needed to do the inspection.  Some customers want it to be part of the system so they can access it in the future as they get better or need the capability.  The customers get to choose from a menu of capabilities and can make the system as robust as desired.  Everything has a price, however.’‘

Larsen: ‘‘We find two groups of users are most enthusiastic.  First are the large, sophisticated, multi-plant operators.  These customers generally have information networks into which they would also like to tie their machine vision equipment.  They want the ability to monitor equipment performance and extract process-related information.  Second are the small, remote users.  Service calls to far, out-of-the-way places are very expensive -- particularly for a user that may have only one piece of machine vision equipment.  Experience has shown that some personal service calls can be avoided entirely by remote diagnosis and resolution of a problem.  Not only does the customer save significant travel and labor expenses, but the equipment is operating sooner reducing losses due to down time.’‘

Wintriss: ‘‘If they don't ask, they should.  It provides fantastic user interface possibilities.’‘

Are there industries that you find discourage this capability? Why?
Toth: ‘‘Many companies in certain industry segments disallow remote diagnostics by their internal policies. These policies restrict any possibility of leaving their facility, or even a designated area within the plant. In such places remote diagnostics is simply strictly excluded. Large companies sometimes create their own internal remote diagnostics service center using the built-in remote tools. The same facility is also used to collect production and maintenance data for statistical data analyses to find out and make sure that main parameters are in line and the company is not ‘bleeding.’ In some industry segments these data are then also entered into an industry database anonymously.  Members who provided the data will receive industry min, max and average production data for sub- processes, so they can compare continuously, how they perform relative to other manufacturer. Smaller companies cannot afford such a facility because of obvious reasons. The ‘tele-service’ facility is also used for remote training, consultation and support, when the remote service technician is able to see the same screen as the operator on the field.’‘

Wintriss: ‘‘Sometimes people are somewhat afraid of opening up their machines to the web.  Usually a separate net, apart from the main company network is OK.’‘

Willis: ‘‘Remote troubleshooting has many security issues, especially with validation and process control being such important issues for the pharmaceutical industry.’‘ 

Mathiszik: ‘‘More elaborate concepts of dialing into equipment on the customer's floor for remote diagnostics have met resistance due to fire wall/security restrictions.’‘

Blackwell: ‘‘People are skeptical across all industries. There are concerns with plant floor production networks. There’s an unwillingness to provide access to secure corporate networks for fear of increasing firewall vulnerability, risking corporate intellectual property, and loosing competitive advantage.’‘

Larsen: ‘‘We still find considerable reticence among many of our customers in enabling and utilizing this capability.  Some customers have serious concerns about information and network security.  Even though security measures can be very effective, some customers perceive a risk they simply don't want to deal with.  Other customers believe that because we supply reliable equipment, they will not get enough benefit from the few problems that may arise, compared to the cost of installing and maintaining a data connection to the equipment.  Still other customers believe they get more complete service, and better results with a traditional on-site service call.’‘

Reardon: ‘‘I wouldn't say ‘discourage.’ I would say, ‘They are not inclined to pay for the service.’ They run on such slim margins that any up-front expenditure that prevent future costs/losses are questioned and usually refused. They tend to manage their businesses from crisis to crisis. ’‘

Goeckel: ‘‘Yes, there are industries that either do not want you to get that close to their operation or don't want you to have access to the information.  Some customers do not let you into their network and also do not want you to ‘manage’ their process for them.  In the hygienic industry, for example, most customers don't even want you to know how they are using the equipment.  They dictate what they want and you deliver.  There are even a few customers who do not want anyone to know they have your equipment.  We often sign an agreement to maintain confidentiality.  It is all about maintaining a competitive edge.’‘

What are the advantages to the machine vision user of this approach to servicing a system?
Goeckel: ‘‘It is cheaper and problems are usually solved more quickly.’‘

Mathiszik: ‘‘Fast response time. Plugging into shared worldwide knowledge. MVP's global support staff are all copied on support requests, and a lot of positive input from across the globe help in fast response and an ability to deliver ‘best in class’ solutions world-wide.’‘

Larsen: ‘‘The speed of response can be incredibly fast.  Where downtime can be very costly, any time advantage in getting the machine vision equipment back in operation has great economic value.  Teleservicing also provides an opportunity for training the operators of the equipment - this can be particularly valuable in bridging knowledge gaps created by turnover among equipment operators.’‘ 

Reardon: ‘‘1) He doesn't need to be an expert in machine vision to be successful; we provide the expertise; this saves him training costs and increased labor costs; 2) It minimizes downtime, which in our industry is frequently more important than in many others; 3) It can optimize the performance of his system more consistently, which results in increased yield, quality or both.’‘

Blackwell: ‘‘One advantage is potentially eliminating travel costs and time delays associated with conventional on-site support. As a result, customers could have faster start-up times and less down time. Also, remote demonstrations that walk customers through the solution are more effective for people that learn better by example.’‘

What are the advantages to the machine vision supplier of this approach to servicing a system?
Willis: ‘‘OEM can see the problem/reach a solution from offsite location.’‘

Balckwell: ‘‘Suppliers and users alike share the advantages.’‘ 

Larsen and Reardon: ‘‘We hopefully get more frequent contact with our customer and our equipment to ensure that equipment performance is up to expectation and that the customer remains satisfied.  We can also provide more value-added service for our customers with less wear and tear on our service team by reducing wasted travel time and expense.  Remote monitoring can also help use reduce warranty expense through rapid, efficient diagnosis of the problem. And we can support our customers, especially those who are more remote from our service techs, at much lower cost.’‘

Goeckel: ‘‘The machine vision supplier does not have to send out a technician which is not the most profitable area for the company.  You don't make money on servicing but rather developing, building and selling systems.  Since most companies are relatively small and the engineers wear several hats, it is more profitable to spend time in these areas rather than servicing.  Also, most customers get a one-year warranty with the system, so during that first year he claims the problem is with the system and wants the servicing free.’‘

Mathiszik: ‘‘Fast response, less travel cost, ‘closeness’ to the field.’‘

Wintriss: ‘‘Big money saver.  Allows us to download updates to software running in both the cameras and the controller PC.  Also allows us to download changes to the Blob Engine which runs in the FPGA in our cameras.’‘

What does this approach to servicing a system do for the lifecycle cost of the system?
Blackwell: ‘‘It could reduce the lifecycle cost of a system by providing real-time installation, set up, and development assistance to reduce start-up time. Live support could also reduce downtime, and lower the cost of diagnostics by perhaps eliminating the need to ship units back for troubleshooting.’‘

Wintriss: ‘‘Big reduction in lifecycle cost...both to customers and suppliers.’‘

Goeckel: ‘‘It extends the life of the system.  Many customers are running 10-year-old systems that still work.  The only way to get them to buy a new one is if they need additional capability or you tell the customer you are no longer going to support the old system.  This usually irritates him and he goes out and buys from competition.  So you have to be very careful how you approach the replacement concept.’‘

Larsen: ‘‘Presumably the lifecycle cost will be reduced.  However, we see a greater economic benefit to users through improved equipment performance.  Our machine vision equipment has a direct effect on yield, quality, and cost.  Optimal performance of our equipment, facilitated by teleservicing, will bring economic value to our customers through improved yield and quality and reduced cost.’‘ 

Mathiszik: ‘‘Less support cost compared to service model without remote assistance capability.’‘

If you do not offer this capability are there arguments for not offering it today?
Goeckel: ‘‘We offer this capability and it is one of our strengths because it gets you repeat orders.’‘

Blackwell: ‘‘Corporate network security and firewalls complicate remote access. Even though some vision systems are networked internally, most aren’t connected directly to the Internet. Access to our hardware is limited in OEM applications because the vision system is so tightly integrated into a much larger system.’‘