Data Streaming Architectures in PXI Systems

Publish Date: Oct 23, 2013 | 13 Ratings | 3.38 out of 5 |  PDF

Overview

Traditionally, instrumentation systems have implemented limited data streaming. Stand-alone instruments interface to controlling PCs through buses that lack the bandwidth to sustain transfer rates, allowing the instruments to continuously acquire or generate at a high sample or update rate. For example, the majority of acquisitions performed with stand-alone oscilloscopes are finite. The duration of the acquisition is dictated by the amount of onboard memory available in the oscilloscope. After the acquisition is complete, the data is transferred to the controlling PC. In the same manner, with a stand-alone arbitrary waveform generator, the waveform is downloaded to the device’s onboard memory, and this single waveform is generated once or looped from device memory.

Because it is based on the high-bandwidth PCI and PCI Express buses, the PXI platform enables instruments to stream data to or from sources other than onboard device memory. Considering the previous example, a PXI oscilloscope is able to continuously acquire at a high sample rate because the data can be transferred in real time from onboard device memory across the PCI or PCI Express bus at a rate high enough (up to 2 GB/s) to guarantee that data is not overwritten before it is transferred from device memory.

Before examining the data streaming architectures that you can implement in PXI systems, it is important to understand how streaming takes place on both the PCI and PCI Express buses.

Table of Contents

  1. Streaming Data with the PCI Bus
  2. Streaming Data with the PCI Express Bus
  3. Device Memory
  4. Controller Memory
  5. Controller Hard Drives
  6. Direct-to-Disk Controller
  7. Relevant NI Products and Whitepapers

1. Streaming Data with the PCI Bus

PCI is a parallel bus. The most common implementation, and that which is used in PXI, is 32-bits wide with a 33 MHz clock. This results in a theoretical maximum bandwidth of 132 MB/s (approximately 110 MBytes/s can be sustained). Because the bus is parallel, all of the devices on the bus share its bandwidth. Data that is acquired by a PCI device is transferred from onboard device memory across the PCI bus, through the PCI controller, across the I/O bus, and into system memory (RAM). It can then be transferred from system memory, across the I/O bus, onto a hard drive(s). The CPU is responsible for managing this process. Data that is generated by a PCI device follows the opposite path. Additionally, peer-to-peer data streaming between two devices on the PCI bus is possible. Figure 1 illustrates the data-streaming architecture of a PCI-based system, such as PXI.


Figure 1. Data streaming architecture of a PCI-based system, such as PXI

Back to Top

2. Streaming Data with the PCI Express Bus

PCI Express, an evolution of the PCI bus, maintains software compatibility with PCI but replaces the parallel bus with a high-speed (2.5 Gbits/s) serial bus. PCI Express sends data through differential signal pairs called lanes, which offer 250 MBytes/s of bandwidth per direction per lane. Multiple lanes can be grouped together to form links with typical link widths of x1 (pronounced "by one"), x4, x8, and x16. A x16 link provides 4 GBytes/s bandwidth per direction. Moreover, unlike PCI, which shares bandwidth with all devices on the bus, each PCI Express device receives dedicated bandwidth. PXI Express supports up to 6 GBytes/s of total system bandwidth (controller to backplane) and up to 2 GBytes/s of dedicated bandwidth per slot (backplane to module).

Data that is acquired by a PCI Express device is transferred from onboard device memory across a dedicated PCI Express link, across the I/O bus, and into system memory. It can then be transferred from system memory, across the I/O bus, onto a hard drive(s). The CPU is responsible for managing this process. Data that is generated by a PCI Express device follows the opposite path. Peer-to-peer data streaming is also possible between two PCI Express devices. Figure 2 illustrates the data-streaming architecture of a PCI Express-based system, for example PXI Express.


Figure 2. Data streaming architecture of a PCI Express-based system, for example PXI Express


PCI and PCI Express are high-bandwidth buses that enable numerous data streaming architectures to be implemented in PXI instrumentation and data acquisition systems. These architectures include the following:

  • Device memory
  • Controller memory
  • Controller hard drives (including RAID arrays)
  • Direct-to-disk controllers

Back to Top

3. Device Memory

As discussed previously, streaming acquired data from an instrument to onboard device memory or generating data from onboard device memory to an instrument is the most common data streaming architecture. Because the memory is local to the device, and data does not need to be transferred across a bus, it can be written to and read from at the full sample or update rate of the instrument. Additionally, because the data streaming is performed by an onboard FPGA or ASIC, minimum processing is required of the system controller (see Figure 3). The application programming interface (API) support for this data streaming architecture is, most often, straight-forward and simple because it is a common use case.


Figure 3. Functional block diagram of a PXI oscilloscope from National Instruments


The main disadvantage of streaming data to and from device memory is the limited depth. The limited depth requires the balancing of sample or update rate and recording or playback duration. PXI modules are available from National Instruments with up to 512 MBytes of memory per channel. With an 8-bit digitizer acquiring at 200 MSamples/s, the memory will be filled in just over 2.5 seconds. Device memory, because of the design challenges associated with integrating it into an instrument is also more expensive than standard PC memory.

Back to Top

4. Controller Memory

The memory (RAM) available in the PXI system controller has a greater depth than onboard device memory. Up to 16 GBytes of memory can be used with an embedded controller or in a PC remotely controlling a PXI system. This memory is less expensive than device memory because it is standard PC RAM.

Streaming to controller memory takes place from device memory, across the PCI or PCI Express bus, through the PCI controller (in the case of PCI), and across the I/O bus. Streaming from controller memory takes place in the opposite direction. With a PXI module, the PCI bus limits the data streaming rate to approximately 110 Mbytes/s. With a PXI Express module, the maximum data streaming rate is dictated by the component (chassis, controller, or module) with the minimum PCI Express link width (see Figure 4). For example, in a system composed of a x4 chassis, x4 controller, and x1 module, the module will limit the maximum data streaming rate of that particular slot to 250 Mbytes/s. Some processing is required of the controller to manage the transfer of data from device memory to controller memory for acquisition or from controller memory to device memory for generation.


Figure 4. With PXI Express, the maximum data streaming rate of a particular slot is dictated by the component (chassis, controller, or module) with the minimum PCI Express link width

Back to Top

5. Controller Hard Drives

PXI controller hard drives provide deep and inexpensive storage for streaming data to or from instruments or data acquisition modules (commonly known as streaming to disk and streaming from disk). The hard drive(s) can be internal to an embedded controller, rack-mount controller, or PC remotely controlling a PXI system, or it can be external to the system and interfaced to through a PXI or PXI Express peripheral module or an ExpressCard module placed in the ExpressCard slot available on many embedded controllers (see Figure 5). With either a PXI Express or ExpressCard module, the PCI Express bus provides the interface between the I/O bus and hard drive(s).


Figure 5. Many PXI and PXI Express embedded controllers include an ExpressCard slot that can be used to interface to external hard drives using the PCI Express bus


Streaming to controller hard drives takes place from onboard device memory, across the PCI or PCI Express bus, through the PCI controller (in the case of PCI), across the I/O bus, through controller memory, and across the I/O bus. Streaming from controller memory takes place in the opposite direction. Processing is required of the controller to manage the transfer of data from device memory to controller memory to hard drives for acquisition or from hard drives to controller memory to device memory for generation. Regardless of whether the hard drives are internal or external, that rate at which data can be written to or read from the hard drives limits the data streaming rates. One way to increase this rate is to use a RAID-0 array of multiple hard drives. RAID stands for redundant array of independent disks. A RAID-0 array increases the rate at which data is written to and read from hard drives by evenly distributing data among multiple hard drives (see Figure 6). As an example, with the National Instruments PXI-8351 or PXIe-8351 rack-mount controller configured with a RAID-0 array of two SATA II hard drives, data can be streamed to disk from instruments of data acquisition modules at up to 80 Mbytes/s.


Figure 6. A RAID-0 array increases the rate at which data is written to and read from hard drives by evenly distributing data among multiple hard drives

Back to Top

6. Direct-to-Disk Controller

When streaming to or from controller memory or hard drives in a PXI system, the PCI controller (in the case of PCI), I/O bus, controller memory, and processor are shared data paths, which divides down data streaming bandwidth. Additionally, the operating system, drivers, and application software managing data flow introduce latencies. Moreover, the operating system’s management of file I/O is not deterministic.

To overcome these limitations, a direct-to-disk controller can be paired with PXI or PXI Express instruments and data acquisition modules to directly and deterministically record or playback data to/from an array of hard drives at sustained, high rates for long durations. With a PXI or PXI Express direct-to-disk controller module, data is streamed directly from the device memory onboard an instrument, across the PCI or PCI Express bus, to the direct-to-disk controller for acquisition (see Figure 7). For generation, data is streamed directly from the direct-to-disk controller module, across the PCI or PCI Express bus, to the device memory onboard the instrument.



Figure 7. Direct-to-disk controllers can directly and deterministically record or playback data to/from an array of hard drives at sustained, high rates for long durations


One example of a direct-to-disk controller for the PXI platform is the Conduant StreamStor. A StreamStor PXI-8108 module can be paired with a PXI instruments to directly record or playback data at a sustained rate of 110 Mbytes/s. For an acquisition, the instrument driver for the PXI instrument writes data from the device memory across the PCI bus directly to a memory address mapped to the StreamStor controller, which then deterministically writes the data to a bank of up to eight 400 GByte SATA hard drives for a total storage capacity of up to 3.2 terabytes of data (generation takes place in the opposite direction). Figure 8 illustrates a PXI instrumentation system with a StreamStor direct-to-disk controller.


Figure 8. PXI instrumentation system with a Conduant StreamStor direct-to-disk controller


The PXI platform, based on the high-bandwidth PCI and PCI Express buses, enables instruments to stream data to or from sources other than onboard device memory, such as controller memory, controller hard drives, and direct-to-disk controllers. With up to 2 GBytes/s of bandwidth available to individual instruments, PXI enables data streaming applications that are not possible with stand-alone instruments.

Back to Top

7. Relevant NI Products and Whitepapers


National Instruments, a leader in automated test, is committed to providing the hardware and software products engineers need to create these next generation test systems.

Software:

Hardware:

Test System Development Resource Library

National Instruments has developed a rich collection of technical guides to assist you with all elements of your test system design. The content for these guides is based on best practices shared by industry-leading test engineering teams that participate in NI customer advisory boards and the expertise of the NI test engineering and product research and development teams. Ultimately, these resources teach you test engineering best practices in a practical and reusable manner. Download guides from the Test System Development Resource Library.

Back to Top

Bookmark & Share

Ratings

Rate this document

Answered Your Question?
Yes No

Submit