An Introduction to Peer-to-Peer Streaming

1. Introduction to Peer-to-Peer Streaming

NI peer-to-peer (P2P) streaming technology uses PCI Express to enable direct, point-to-point transfers between multiple instruments without sending data through the host processor or memory. This enables devices in a system to share information without burdening other system resources. NI P2P technology is supported on PXI Express NI FlexRIO field-programmable gate array (FPGA) modules (NI PXIe-7961R, PXIe-7962R, PXIe-7965R, and PXIe-7966R) and PXI Express digitizers and vector signal analyzers, including the NI PXIe-5122, PXIe-5622, and PXIe-5663.

In the system depicted in Figure 1, an NI PXIe-5622 digitizer in an NI PXIe-1075 chassis uses peer-to-peer data streaming to send data directly to an NI PXIe-7966R NI FlexRIO FPGA module. The FPGA module then sends data to another FPGA module for additional processing. Because the chassis backplane switches provide direct links to the slots occupied by the modules, you do not need to transfer data through the host controller or use system resources such as the CPU or host memory.

Figure 1. An NI PXIe-5622 digitizer streams data to two PXI Express NI FlexRIO FPGA modules.

Back to Top

2. Application Example

One common need in RF applications is a real-time frequency domain trigger. While most RF instruments trigger on a power level, this trigger is independent of frequency. However, with peer-to-peer data streaming and processing using the NI LabVIEW FPGA Module, you can create a frequency-domain trigger. In the application depicted in Figure 2, the NI PXIe-5663 vector signal analyzer uses peer-to-peer streaming to send data to the NI FlexRIO FPGA module, where it is windowed, converted to the frequency domain, and then compared against a mask. When the data exceeds this mask, the FPGA module asserts a digital trigger on the PXI backplane. Once the NI PXIe-5663 receives this trigger, it uses its normal acquisition memory to capture a record of data, including pretrigger samples. You can then access this record from the host through the NI-RFSA driver for additional processing or storage.

Figure 2. In this application, an NI FlexRIO FPGA module adds a frequency-domain trigger to the NI PXIe-5663 vector signal analyzer.

Back to Top

3. Programming Peer-to-Peer Systems

Programming peer-to-peer data streaming is greatly simplified by the NI-P2P driver. In the application depicted in Figure 3, FPGA No. 1 sends data directly to FPGA No. 2. In LabVIEW FPGA, simple peer-to-peer reader and writer nodes provide a first-in-first-out (FIFO) based interface for data exchange. These nodes are similar to DMA and local FPGA FIFOs. Before data exchange is possible, the host must connect the writer stream on FPGA No. 1 to the reader stream on FPGA No. 2 through the NI-RIO and NI-P2P APIs on the host (shown in Figure 3). Depending on the configuration, you need only one or two VIs to connect the peer-to-peer streams so that data exchange takes place.

Figure 3. Peer-to-Peer Streaming between Two NI FlexRIO FPGA Modules and the Associated Software

Back to Top

4. Benchmarking Peer-to-Peer Systems

With NI peer-to-peer technology, data streaming rates of more than 800 MB/s are possible in a single direction.  Maximum throughput is dependent on the streaming modules, chassis, and, if the configuration warrants it, the controller. Generally, the lowest of these rates is the maximum possible P2P bandwidth.

Peer-to-peer transfers are designed to have a very low latency, but it will vary depending on the system configuration.  When streaming only through FIFOs and the PCIe bus the typical latency is around 2-4 μs, though this will likely have occasional an spike into the tens of μs due to other bus traffic.  When streaming from a digitizer we can usually expect around 5 μs of latency and when streaming to an arbitrary waveform generator around 10-20 μs of latency due to onboard buffering.

When streaming to/from devices with onboard signal processing, such as the NI PXIe-5622 digitizer, the additional filters and other processing can add up to hundreds of μs of latency.  Finally, it is important to also consider the latency of the user code operating on the NI FlexRIO FPGA module when evaluating the performance of a system.

Chassis

All data must pass through a chassis to stream from one module to another, so the chassis plays a critical role in determining bandwidth. The PCI Express switches on the chassis backplane route data through the chassis and provide the high-bandwidth point-to-point connections that enable peer-to-peer data streaming. When modules are in chassis slots that are directly connected to the same PCI Express switch, as in Figure 4, the bandwidth is dependent on the switch.

Figure 4. Module placement in the chassis routes all data through a single PCI Express switch.

Table 1 shows the maximum bandwidths of the PCI Express switches in a given chassis. This bandwidth is achievable through any two slots connected to the same switch, and multiple connections per switch are supported at the given rate.

Table 1. PXI Express Chassis P2P Bandwidth

Controller

When modules in a P2P streaming system are not both connected to the same PCI Express switch on a chassis backplane, then data must pass through the host controller’s onboard switch or chipset, but not through its CPU or memory. This configuration is shown in Figure 5.

Figure 5. Module placement in the chassis routes data through the host controller.

Table 2 lists the maximum bandwidths of PCI Express switches or chipsets for a given controller. Note that this number represents the aggregate bandwidth through a certain segment into and out of the controller. If multiple peer-to-peer streams exist over these same segments, they must share bandwidth.

Table 2. PXI Express Controller P2P Bandwidth

Modules

Provided the chassis and controller configuration supports a certain bandwidth, the PXI Express modules themselves can determine the maximum bandwidth achievable. Below are benchmarks and configuration details for various P2P-capable devices.

NI PXIe-7966R, PXIe-7965R, PXIe-7962R, PXIe-7961R NI FlexRIO FPGA Modules: These PXI Express NI FlexRIO FPGA modules are capable of streaming data at more than 800 MB/s into or out of the module. When streaming in both directions simultaneously, the FPGA modules can achieve rates of more than 700 MB/s per direction, or a more than 1.4 GB/s aggregate data rate. This data may consist of a single stream, or up to 16 separate streams, one for each DMA channel of the device. The number of streams and their bandwidths are determined by the configuration and programming of the FPGA on the device.

NI PXIe-5622 IF Digitizer: The NI PXIe-5622 is a 150 MS/s, 16-bit digitizer that can produce data at up to 300 MB/s. For peer-to-peer streaming, the digitizer has a single writer endpoint that is located on a parallel data path to the onboard memory. You can write the acquired data to this endpoint to stream to an FPGA target as well as send it to the host through onboard memory. A key feature of the NI PXIe-5622 is the onboard signal processing (OSP) with quadrature digital downconversion (DDC), which provides up to 60 MHz of IF bandwidth in the form of complex I/Q data at 75 MS/s, or 250 MB/s. In this mode, the samples are interleaved by sample in the peer-to-peer stream, with the I sample followed by the Q sample.

Figure 6. Peer-to-Peer Streaming with the NI PXIe-5622 IF Digitizer

Peer-to-peer data streams flow in the raw binary format, which does not include scaling or calibration. The normalization coefficients can be queried by NI-SCOPE and applied to calibrate the binary data without scaling to volts. The normalized data adheres to the following criteria, so you can later interpret the binary information or scale it to volts.

• The maximum positive binary value maps to the maximum positive voltage of the vertical range.
• The maximum negative binary value maps to the maximum negative voltage of the vertical range.
• The vertical range is divided evenly across the possible binary values.

Alternatively, you can query and apply the scaling coefficients to calibrate and scale the data in a single step. Refer to the High-Speed Digitizers Help for details.

NI PXIe-5122 High-Resolution Digitizer: The NI PXIe-5122 is a 100 MS/s, two-channel, 14-bit digitizer that can produce data at up to 200 MB/s per channel per stream. For peer-to-peer streaming, the digitizer has two writer endpoints that are located on parallel data paths to the onboard memory. You can write acquired data to these endpoints as well as send them to the host through onboard memory. 

Figure 7. Peer-to-Peer Streaming with the NI PXIe-5122 High-Resolution Digitizer

Examples of valid streaming configurations are shown in Table 3:

Table 3. Valid Streaming Configurations

Note that it is possible to configure the module to generate data at rates that exceed the PXI Express x4 bus bandwidth, which can result in data overflow. For example, the last configuration of Table 3 results in 400 MB/s per stream when sampling at the full rate (800 MB/s total). If you also send data to the host PC, you exceed the bus bandwidth. Decreasing the digitizer sampling rate can prevent data overflow.

Back to Top

5. Calibration

Peer-to-peer data streams are in the raw binary format, which does not include scaling or calibration. The normalization coefficients can be queried by NI-SCOPE and applied to calibrate the binary data without scaling to volts. The normalized data adheres to the following criteria, so you can later interpret the binary information or scale it to volts.

• The maximum positive binary value maps to the maximum positive voltage of the vertical range.
• The maximum negative binary value maps to the maximum negative voltage of the vertical range.
• The vertical range is divided evenly across the possible binary values.

Note: The maximum and minimum binary values span the NI-SCOPE programmed vertical range (e.g. ±11 V) even if the user has specified ±10 V.  This is to capture overshoots, even though the user set the range to be less than the overshoots.

Alternatively, you can query and apply the scaling coefficients to calibrate and scale the data in a single step. Refer to the High-Speed Digitizers Help for details.

Back to Top

6. Supported Devices

The following list of National Instruments hardware supports P2P streaming:

Table 4.  National Instruments P2P Supported Hardware

* The above mentioned RF Signal Analyzers are composed of three components; a RF downconverter, a local oscillator, and a digitizer.  The digitizer is the component that does the P2P streaming.  The digitizer that is used in these RF Signal Analyzers is the PXIe-5622.  Similarly, the arbitrary waveform generator performs the P2P streaming for the RF Signal Generators.

**The PXIe-5122 and PXIe-5622 must have a current firmware version to enable Peer-to-Peer streaming, which was enabled as of NI-SCOPE 3.6. Follow the steps in the NI-SCOPE 3.6 Readme to learn how to update the firmware on your PXIe-5122 or PXIe-5622 device.

Back to Top

7. Example Code

• Peer-to-Peer NI RF Signal Analyzer to NI FlexRIO FPGA Module
• Multi Record Analog Trigger using NI-Scope, NI-P2P, and NI FlexRIO
• Working with Interleaved Data using NI-Scope, NI-P2P, and NI FlexRIO