NI-DAQmx High-Speed Streaming to Disk

Publish Date: Jan 09, 2019 | 3 Ratings | 3.00 out of 5 | Print | Submit your review

Overview

The TDMS file format is the NI recommended file format for saving time-based measurement data to disk for high performance, usability, and reuse. With NI-DAQmx 9.0, TDMS file writing is now directly integrated into the NI-DAQmx driver for the easiest and fastest way to stream data to disk reaching speeds up to 1.2 GB/s.

Table of Contents

  1. NI-DAQmx High-Speed Logging
  2. Optional Data Compaction
  3. Additional Resources

 The National Instruments Getting Started with NI-DAQmx Series is aimed at helping you learn NI-DAQmx programming fundamentals. Through video and text tutorials, this series will take you from verifying your device's operation in Measurement & Automation Explorer (MAX) to programming data acquisition applications using LabVIEW. It is intended for both the beginner who wants to learn how to use the DAQ Assistant, as well as the experienced user who wishes to take advantage of advanced NI-DAQmx functionality.


1. NI-DAQmx High-Speed Logging

NI-DAQmx 9.0 or later installs a new VI, DAQmx Configure Logging VI, which allows users to configure writing data to a TDMS file within a single VI on the DAQ palette.  Not only does this make it simple to use and integrate into existing data acquisition applications, but with under the hood optimizations, it is the fastest way to write data to disk.

Figure 1. The DAQmx Configure Logging (TDMS) VI allows users to set up data logging to TDMS before even starting the task.

To optimize file writing speeds, NI-DAQmx can stream data directly from the device buffer to the hard disk bypassing the Windows, LabVIEW, and TDMS buffers. The Voltage -Continuos Input.vi also improves performance and reduces the disk footprint by writing raw data to the TDMS file, including the scaling information separately for use when reading back the TDMS file. These optimizations result in writing speeds up to 1.2 GB/s. The example can be found by selecting Help>>Find Examples... that opens the NI Example Finder from there select Hardware Input and Output>>DAQmx>>Analog Input>>Voltage - Continuous Input.vi to open the example.

Figure 2. NI Example Finder.

Once the Data has been stored to the TDMS file, you use the TDMS File API and TDMS File Viewer in LabVIEW, DIAdem, TDMS Excel Add-on, or other TDMS supported interfaces for reading and viewing the acquired data. Learn more about TDMS writing and reading interfaces.


Back to Top

2. Optional Data Compaction

The primary factors which affect high-speed streaming to disk in a PC-based data acquisition system are disk I/O bandwidth, CPU bandwidth, and bus bandwidth. Assuming the professional developer has sufficient bandwidth for both the disk I/O as well as the bus, NI-DAQmx data compaction can provide significant performance gains for high-speed streaming to disk applications.

NI-DAQmx primarily affects streaming-to-disk performance because it uses CPU bandwidth while moving and scaling data acquired by devices. Therefore, on systems where CPU bandwidth is a limiting factor, it is critical that NI-DAQmx efficiently uses the CPU.

NI-DAQmx implements an extremely simple form of data compaction. Bits at the end of samples are simply thrown out and the resulting samples are packed into memory. Developers configure NI-DAQmx to discard one or more of the least significant bits in a sample. For example, you can configure a channel with 24-bit resolution and a 32-bit sample size to return only the 20 highest-order bits. A total of 12 bits are discarded, including eight unused bits, which reduces the required disk space by 37.5 percent. In some cases, no data is “lossed” because the samples already contain unused bits. For example, a channel with 12-bit resolution and a 16-bit sample size contains four unused bits. You can discard the four bits with no loss while reducing the required disk space by 25 percent. NI-DAQmx does not implement any other lossless or lossy compression algorithms such as LZW or JPEG.

Figure 3. Use NI-DAQmx data compaction to improve high-speed streaming to disk performance.


Back to Top

3. Additional Resources

 

 

Back to Top

Bookmark & Share


Ratings

Rate this document

Answered Your Question?
Yes No

Submit