From Fashion to Football, Vision Pushes the Envelope of Entertainment

“All hell broke loose” when digital imaging came to the movie industry in the 1980s, according to Joe Cicio of Third Reef Productions (Los Angeles, California), Hollywood cinematographer, tech guru, and special effects specialist. But what Joe didn’t know is that 1985 was only the first step on a very wide road.

From creating 3D replays of sporting events to avatars for online shopping that show you exactly how new clothes will fit on your body, machine vision technology is reshaping all aspects of entertainment. With camera resolutions reaching double digits at 30 frames per second or more and computing power keeping pace, machine vision is blurring the virtual line between what you see and what you get.

A Sporting Chance
In the 1990s, machine vision technologies changed how tennis, cricket, and football fans all looked at controversial referee calls. The Hawk-Eye system, developed in the UK by Dr. Paul Hawkins and now owned by SONY Japan, was the first to take digital cameras, image processing technology, and software originally designed for missile tracking applications and create a system for verifying service faults and out of bounds calls in tennis, as well as leg-before-wicket calls in cricket, according to Rich Dickerson, marketing communications manager at JAI, Inc. (San Jose, California). Using ten high-speed cameras, Hawk-Eye would capture various parameters of a tennis ball in motion, for example, including speed, trajectory, and spin, then project that information to the boundary line of a tennis court. The resulting data is used to create a graphic of where the ball came into contact with the ground in relation to the boundary line. Today, tennis fans are intimately familiar with this useful visual tool. The same technology is used to determine whether a pitched ball would have struck the wicket if not blocked by the batter’s leg in professional cricket matches.

Last year, Israel’s Replay Technologies revealed a new use for machine vision technology at a U.S.-based National Football League (NFL) game between the Dallas Cowboys and New York Giants. Called freeD, the system placed twelve, 12MP Falcon2 cameras from Teledyne DALSA, (Waterloo, Ontario, Canada) around both end zones for a total of 24 cameras. Real-time video from each camera is fed into a processing unit, which takes the 2D information and combines it using image processing and triangulation software to create 3D voxels for the entire field of view. This 3D scene can then be rotated in any direction by the television production staff to reveal hidden features unseen by any one camera, such as where a foot came down in relation to a boundary line. Cicio has used the same compact Falcon2 cameras to capture 4k ultra-high-definition (UHD) video in tough locations, such as on the back of a speeding motorcycle.

“There are a number of ways that companies are creating the 3D data sets for sporting events,” says Steve Kinney, director of technical pre-sales and support at JAI. “Some use many cameras at lower resolutions. This year, one company will use several of JAI’s 20MP cameras to create the same 3D replay effect for the NFL. These are inward- looking systems, but there are also immersion video systems where the cameras look outward, giving the viewer the chance to experience the event — such as a concert — virtually. These cameras are very compact at 29mm-by-50mm without the lens and use GigE output. The long cabling capability helps in these types of applications.”

According to Kinney and Dickerson, companies are already working on new innovations, combining the 3D imaging capability with the ball-tracking capability of other sports video systems to create large interactive environments where people can virtually participate in major sporting events, similar to arcades but on a larger scale than ever seen before. Video game producers are using high-resolution cameras to capture more than the body shape of fans' favorite sports stars. The cameras, along with portable workstations, are being used to create high-resolution 3D models of these athletes’ expressions, down to wrinkles in the skin. “It’s all about increasing the realism in these games, from a person’s face to mannerisms,” Kinney adds.

3D Body Scans Take Guesswork Out of Online Shopping
Vitronic GmbH’s (Wiesbaden, Germany) VITUS^bodyscan full body pillar-based 3D scanner also applies machine vision technology to sport, but from the perspective of the athlete. Depending on the model, Vitronic mounts one or two cameras on up to four pillars. The person steps into the middle of the pillars, and the cameras travel down each pillar, capturing high-resolution images of a laser line projected on the person’s body. Using triangulation algorithms, the system captures a high-resolution 3D image of the person with 1mm resolution. The newest model, out early next year, will cut the scan times from 12 to 6 seconds and include color and texture image acquisition as well.

“The two camera high-resolution system has been used in a number of research studies, including posture studies for athletes to identify when one hip is higher than another and other factors that may lead to poor performance,” explains Julian Martini, Vitronic’s sales manager for the 3D body scanner. “It’s also been used to scan thousands of individuals in several European countries to provide data for clothing retailers so they know what sizes to stock to accommodate the highest percentage of their potential customer group. But mostly, the system has been used in fashion applications through our distribution partner, Human Solutions. Their software can make up to 150 measurements to help people get the perfect fit from their clothes from a single 3D scan. If people create an avatar of themselves from one of these scans, they could confirm that clothes purchased online would fit properly before they buy, reducing returns, which are a huge cost for online retailers.”

According to Martini, the new VITUS^bodyscan system can be used in video production and other animated applications. “There is software on the market that can animate our 3D scans,” Martini says, “this will open up new virtual-reality applications.” The new scanner is also an ideal tool to manufacture 3D-printed figurines. These mini-versions of oneself have become a new means to freeze time. Over the course of the last year we have seen businesses opening up all around the globe which offer these 3D figurines. VITUS^bodyscan captures the 3D scans of people that are subsequently ‘printed out’ by 3D printers.”

With the massive financial resources of the entertainment, fashion, and online shopping industries behind it, machine vision technology can offer a seemingly limitless number and variety of applications to the entertainment industries. The road that Joe Cicio first took some 30 years ago will likely wind on endlessly as people seek new experiences in virtual worlds that are no longer bound by the traditional rules of physics, geography, and time.