Choosing the right file format for saving data in your test system can be a big decision. You need to pick a format that meets your requirements in your current system, but that can also scale as your test needs change. You have many characteristics to consider when evaluating storage formats:
- File sharing and exchangeability
- Disk footprint
- Simple inclusion of meta information and properties
- Reading and writing speeds
- Working familiarity with the format
As each application is different, LabVIEW contains a wide array of file I/O options for you to choose from, including ASCII, binary, datalog, ZIP, and XML.
Figure 1: With LabVIEW, choose from many interfaces for reading and writing files, depending on your needs.
As seen in Figure 1, LabVIEW includes basic functions you can use to build files from scratch as well as Express VIs for configuring file I/O parameters from a dialog so you don’t have to worry about details such as opening file references, string formatting, and endianness. Because good file I/O practices can be one of the trickier parts of building any test or measurement system, NI LabVIEW ships with dozens of examples that you can use to get started with each of the file types.
File I/O Designed Specifically for Engineering Data
Despite the fact that LabVIEW offers a wide variety of file I/O options, these traditional file types rarely meet all the criteria you need in a file format. For example, ASCII files are exchangeable, but are very large and slow to read and write. On the other hand, binary file read and write speeds can keep up with high-speed hardware, but are difficult to share with others.
Because of the drawbacks of traditional file I/O, National Instruments developed the Technical Data Management Streaming (TDMS) file format to meet the specific needs and high demands of engineers and scientists. TDMS files are based on the TDM data model for saving well-organized and documented test and measurement data. The TDM data model offers three levels of hierarchy, as shown in Figure 2 – file, group, and channel. The file level can contain an unlimited number of groups, and each group can contain and unlimited number of channels. Because of this channel grouping, you can organize your data to make it easier to understand. For example, you may have one group for your raw data and another group for your analyzed data within one file, or you may have multiple groups that correspond to sensor types or locations.
Figure 2: The TDM data model meets the specific requirements of measurement data.
Also, you can insert your own custom properties at each of the three levels. Each level accepts an unlimited number of custom-defined attributes to achieve well-documented and search-ready data files. The descriptive information located in the TDMS file, a key benefit of this model, provides an easy way to document the data much like you would document code. As your documentation requirements increase, you do not have to redesign your application, you simply extend the data model to meet your needs.
Multiple Easy-to-Use Programming Interfaces
Because it was developed to meet the needs of all engineers, TDMS offers ease of use, high-speed streaming, and exchangeability. Like many operations in LabVIEW, you can use multiple interfaces to write TDMS files. You can quickly read and write TDMS files using a virtual instrument (VI) such as the Write To Measurement File Express VI or, for the best performance and customization, use the primitive TDMS VIs from the File I/O palette. Also, when using LabVIEW with NI-DAQmx, you can use the Configure Logging VI from the DAQ palette or log directly from the NI DAQ Assistant, as seen in Figure 3.
Figure 3: With multiple interfaces for writing TDMS files, you can choose the one that best fits your needs.
High Streaming Speeds
Although TDMS is designed to be easy to use, with the major speed improvements in hardware over the last several years, it is also important for TDMS to be optimized for high-speed streaming so that file I/O is not the bottleneck when determining your acquisition speeds. With under-the-hood optimizations for bypassing buffering, TDMS is the fastest way to write data on the NI platform. So far, TDMS has been tested to stream data up to 3.6 GB/s.
Files Exchangeable with Other Programs such as Microsoft Excel
Because you may be required to work in additional applications, TDMS is easily exchangeable across other programs. You can open TDMS files in Microsoft Excel using the TDM Excel Add-In, which installs with NI software and is available free at ni.com. You also can use a C DLL for reading and writing TDMS files in other programming languages. NI is committed to helping you write well-organized and documented data using the TDMS file format, regardless of which products you use.