10 GigE Best Practices: latest hardware for multiple 10GigE camera systems
10 Gigabit Ethernet (10GigE) is enabling the next generation of machine vision applications. By delivering 10Gbit/sec of throughput over a reliable and widely available interface, 10GigE cameras like the FLIR Oryx build on the strengths that make GigE the dominant machine vision interface. The speed and simplicity of 10GigE gives developers the freedom to design the next generation of multi-camera machine vision systems. This article provides best practices for combining multiple GigE and 10GigE cameras into a single vision system.
Setting up your PC for multiple 10GigE cameras
When setting up a multiple 10GigE camera system, the most important host-side specifications are the Network Interface Card (NIC) and the system memory.
The best network interface cards in FLIR tests
In FLIR’s evaluations, Tehuti-based NICs with a Marvell or Aquantia physical layer (PHY) deliver the most reliable performance. To ensure full PCIe bandwidth availability, NICs should be plugged into slots that do not share bandwidth with onboard peripherals like USB ports or audio systems. The slot closest to the CPU is often the best choice; consult your motherboard’s documentation for confirmation of this.
System memory - 10Gbit/sec is a lot of data
Before the CPU makes incoming image data available for application-specific processing, it reassembles ethernet data packets into image frames. This process only uses about 1% of a mid-range i7 CPU’s computing power, but requires enough memory to support the copy operations that arrange data packets in order and read their contents into a host-side image buffer. 10Gbit/sec is a lot of data, necessitating high memory bandwidth. DDR4 memory is recommended and is supported by current Intel and AMD CPUs. A system configuration that supports dual-channel memory is critical.
Dual-channel memory is easy to set up. Instead of using a single-memory module with the desired capacity, install two modules with half the required capacity each into separate memory channels. All modern motherboards support dual-channel memory and many enthusiast ones support triple-channel memory. Intel i7 CPUs are recommended for systems running multiple 10GigE cameras because they support higher memory bandwidth than i5 processors. While grabbing frames from a 10GigE camera is not CPU intensive, the machine vision applications operating on that data may require a large amount of CPU or GPU computing power.
Streaming to disk
A vision system with multiple 10GigE cameras operating at full speed generates a very high volume of data. For applications requiring large numbers of frames saved to disk, NVMe solid-state drives (SSDs) are recommended as they support up to seven times the data rates of SATA III-based storage. The Samsung 970 Pro is the fastest commercially available 1TB capacity SSD. Keeping up with two or more FLIR Oryx cameras running at full speed requires a RAID array of NVMe SSDs. Motherboards are available with controllers for NVMe RAID using M.2 SSDs. The Gigabyte Z170X-SOC supports up to three M.2 NVMe drives enabling write speeds up to 24Gbit/sec. A RAID 0 configuration provides the greatest write speed performance and storage capacity while minimizing system cost and complexity.
The adoption of 10 Gigabit Ethernet as a machine vision interface is driving the development of pre-built systems supporting multiple 10GigE cameras. For example, the Vecow ECS-9700 industrial PC has four 10GBASE-T ports.
Setting up your network for 10GigE cameras
10GBASE-T network hardware is widely available. Hardware supporting 9K or 15.5K jumbo packets maximizes the performance of your 10GigE vision system by reducing network overhead and simplifying the host-side packet reassembly process. For multi-camera systems, FLIR recommends CAT6A cables to minimize cross-talk between adjacent cables.
Not all multi-camera systems require one host-side port per camera. Systems that trigger cameras at lower than maximum rate might share a single host-side 10GigE port between two or more cameras using a switch. The camera triggering sequence must be considered. If multiple cameras trigger simultaneously, transferring two or more images at the same time introduces latency that increases proportionally with the number of cameras triggering.
The availability of affordable 10GigE switches provides systems designers with a simple way to enable IEEE1588 Precision Time Protocol (PTP) on multi-camera systems. PTP can create a common time base across cameras by synchronizing their clocks or to a GPS grandmaster clock connected to the same switch.
A recent trend in 10GigE switches is the addition of combination ports supporting 10GBASE-T and SFP connectors. Switches like the QNAP QSW-1208-8C give system builders unprecedented flexibility. They can connect 10GBASE-T cameras to existing SFP-based hardware without the need for new transceivers. They can also serve as cost-effective media converters enabling multiple 10GBASE-T cameras to operate at km cable lengths over optical fiber, or in environments with high levels of electromagnetic interference (EMI).
Setting up your cameras
When sharing a single port between multiple cameras, system designers must ensure the bandwidth requirements of the camera do not exceed that of the host-side port it connects to. There are several ways to do this, including:
- Selecting a smaller Region of Interest (RoI) to reduce the number of pixels per frame.
- Limiting the frame rate of the camera to reduce the number of frames per second.
- Switching to a lower bit depth pixel format.
- Switching color cameras from RGB pixel format to BAYER8 and enabling host-side color interpolation to reduce the number of bits per pixel.
Fig. 4. When sharing a single host-side port between multiple cameras, bandwidth must me managed to ensure the available bandwidth is not exceeded.
Locking the maximum throughput of your cameras is a useful way to remove guesswork while ensuring their best performance. Locking maximum throughput ensures you know exactly how much bandwidth is available on each camera. In FLIR’s Spinnaker SDK, this control is available under the Device Link Throughput setting.
Combining 10GigE and GigE cameras
FLIR 10GigE cameras are compatible with the GigE Vision protocol, enabling out-of-the-box support for a wide range of third-party software. 10GBASE-T ports auto-negotiate to standard GigE speed when connected to GigE hardware. This compatibility makes mixing and matching 10GigE and GigE cameras on a 10GBASE-T switch and a common SDK easy. By seamlessly combining GigE and 10GigE cameras, designers can optimize imaging performance at different points to deliver accurate and cost-effective systems.
10GBASE-T hardware can greatly simplify predominantly GigE-based systems. Designers can connect GigE cameras to a 10GigE switch and operate them at full frame rate off a single host-side 10GigE port.
The many similarities between 10GigE and GigE cameras make them quick and easy to set up in both single and multi-camera systems. 10GBASE-T networking hardware is widely available. Compatibility with the GigE Vision protocol ensures support for a wide range of software. Best practice resources like this article, as well as many how-to guides, are easily accessible.