Embedded Vision Systems Fundamentals: A Basic Overview of Design and Functionality

Understanding Embedded Vision Components and System Configurations

Embedded vision is an exciting new technology with incredible potential in a wide range of industries, only recently made possible through the miniaturization of vision and processing components. Broadly speaking, an embedded vision system combines image capture and image processing capabilities into one system. However, embedded vision systems come in many different configurations.

At the most basic level, an embedded vision system includes a sensor or camera module, a standard interface and some form of processing architecture. There are a variety of options regarding each of these components for any embedded vision system – choosing the right one depends on understanding design, performance and cost tradeoffs.

To learn how embedded vision systems function, it’s important to take a look at how processing architectures, types of processors, standard interfaces, and vision and imaging modules work to create a whole embedded system.

Embedded Vision Processing Architectures

Powerful, highly compact processing capability is the true enabler of embedded vision – small processing platforms can now do the work that traditionally took large, bulky computers. For embedded vision systems, processing components may take on several different configurations, including system on chip (SoC), system on module (SoM), single board computer (SBC), and fully custom designs.

SoC is a processing architecture where the central processing unit (CPU), graphics processing unit (GPU), interface controller and more are all located on a single chip. The development of this technology was driven predominantly by large mobile device manufacturers, as the cost to develop SoCs can be millions or even billions of dollars.

SoMs actually contain SoCs but also include RAM, power management and bus systems along with the image processing power of the SoCs. SoMs are most often used along with a carrier board with plug connectors for power or network connections. These systems help companies overcome the high costs of SoC development while providing great flexibility for developers to tweak the system to their needs.

SBCs are essentially SoMs that are integrated onto a single circuit board with a series of connectors for peripheral devices. They are SoMs and carrier boards in one – this comes with cost and performance tradeoffs. SBCs have low initial costs and are easy to implement, but given their pre-integrated nature, they don’t have much flexibility to adapt to specific applications.

Fully custom processing designs aren’t common – they’re basically SBCs custom-built for one specific application. They typically arise when a company determines the cheapest option for their embedded vision application is to develop their own SBC. Custom embedded processing architectures are cost-effective as there’s no unnecessary components, but since they’re built for one specific purpose, they provide little to no flexibility in imaging functions.

There are two main types of processing architectures found in embedded vision systems. Even within each of these, there’s variation in functionality between different manufacturers’ proprietary technology. Despite the diversity of architectures, compact image processing forms the foundation of embedded vision technology.

Types of Embedded Vision Processors

An embedded vision system’s image processing capabilities are determined not just by overall processing architecture, but by the type of processor being used. While there are several different types of image processors, the main two types are graphics processing units (GPUs) and field programmable gate arrays (FPGAs).

GPUs are widely deployed in embedded vision systems because of their ability to deliver large amounts of parallel computing potential, especially to accelerate the portions of processing that deal with pixel data. General purpose GPUs (GPGPUs) are one of the most common forms, as they’re built to meet the power constraints for a diverse range of applications.

All GPUs leverage software for imaging algorithms, which comes with advantages and disadvantages. With GPUs, tweaking or altogether changing imaging functions is as easy as changing the system’s software instructions. This provides great flexibility for one system to perform multiple imaging functions. On the other hand, software is always slower than hardware – latency is higher by multiple degrees of magnitude in GPUs.

FPGAs have been gaining favor in recent years for a few reasons. First and foremost, their main advantage is their low latency levels – they are hardware representations of algorithms, making them much faster. FPGAs are also far more energy efficient despite the fact that they provide the highest processing potential with computing performance in the trillions of multiply-accumulates/sec (tera-MACs). FPGAs can simultaneously accelerate multiple portions of a computer vision pipeline – GPUs cannot.

The main drawback of FPGA processors is that they lack the flexibility of GPUs. It takes far longer to prepare a FPGA for another imaging task than tweaking an algorithm in GPUs. However, this has been changing in the past few years as developers have been introducing open source computer vision libraries with pre-configured interface blocks for quicker and easier changes.

While there are certainly other types of image processors, including application-specific integrated circuits (ASICs) and digital signal processors (DSPs), GPUs and FPGAs are two of the most popular. Both have their advantages and disadvantages depending on how they will be deployed in an embedded vision application.

Embedded Vision Standard Interfaces

The connection between the sensor and image processor is important – it can be a bottleneck for performance if chosen incorrectly. As with other embedded vision components, there are several possibilities for embedded interfaces.

USB 3.0 is often a simple plug and play solution for some embedded vision systems, just as they are for traditional machine vision systems. However, they rarely feature a compact design, which immediately rules them out as a solution for many embedded vision system designs that have small space requirements.

Flat flex cables, such as those that follow the MIPI CSI-2 standard from MIPI Alliance, are a more powerful, compact solution for camera integration. However, these types of cables necessitate each individual component at a board-level be adapted to the flat flex cable’s capabilities. This can get costly, and flat flex cables often cannot exceed 20 centimeters in length – two major drawbacks for certain applications.

One of the more reliable interface solutions for embedded vision has been low voltage differential signaling (LVDS) where cables can be several meters long. LVDS cables are typically used with FPGA processors for integration and adjustment of system-specific read-routines. LVDS cables do not use standardized plugs or data transfer protocols, meaning integration costs will be high unless there are openly documented transfer protocols.

The transfer of data between a sensor and processor is a vital process in any embedded vision system. Once again, there are several potential interfaces an embedded system could use, and each has their cost and performance tradeoffs.

Embedded Vision Imaging Modules

The last piece of an embedded vision system is the imaging module. Typically, these are available as sensor modules or camera modules. There’s a subtle difference between the two that has significant consequences for the entire imaging system.

Sensor modules have little to no processing capabilities. They transmit raw image data to a host processor for functions such as noise reduction or application-specific functions like facial recognition. Sensor modules are typically a streamlined, cost-effective approach for embedded vision systems. However, they lack flexibility – a new driver must be coded every time a new sensor is used.

A camera module, on the other hand, includes some processing capabilities along with its image capture responsibilities. This lightens the processing workload on the host processor while eliminating the need for new drivers every time the imaging module is replaced, allowing developers to focus on application software.

Camera modules, typically leveraging FPGAs, can be integrated more easily than sensor modules, but initial costs can be significantly higher than sensor modules. Camera modules recently emerged as an embedded vision peripheral and have proven to be a flexible imaging solution.

The imaging module, whether it’s a sensor or camera module, plays a vital role in the imaging process. Each type performs very different processes which impact overall image quality, energy consumption and costs.

Embedded Vision Systems Are Getting Smaller, Cheaper and More Advanced

Embedded vision systems come in a variety of configurations, but the basic components are generally the same. Embedded system designs are typically streamlined, including just the processing architecture and processor, camera interface, and imaging module.

Understanding the components and system configurations of an embedded vision system is the first step in understanding how an entire embedded vision system works, and how it transforms the applications in which it is deployed. It may seem easy to imagine a system that has a camera, a cable and a processor; but the technology is really very sophisticated.

It is only recently that technological advancements in processors and imaging components have made embedded systems commercially viable. Given their incredible potential in a wide range of industries, developers are constantly working to make embedded vision systems smaller, cheaper and more advanced than ever before.

The future of embedded vision systems looks bright. Several industries will create rapid growth in demand for embedded vision, while others may one day be transformed by the technology. To get an idea of what the future looks like, be sure to read The Future of Embedded Vision.