Introduction to LabVIEW Data Storage VIs

Publish Date: Mar 13, 2013 | 26 Ratings | 4.08 out of 5 |  PDF

Overview

If you have ever struggled to design or maintain a custom file format for measurement data, you can relate to the effort and expense these tasks require. Increasing standards for the information saved during data acquisition require modifying not only the file specification but also all the programs that read and write this format.

Table of Contents

  1. Introduction
  2. LabVIEW Data Storage VIs for Writing, Reading, and Querying TDM Files
  3. TDM Writing Data Basics
  4. TDM Reading Basics
  5. TDM Reading with Query Conditions
  6. TDM Reading a Subset of a Data Channel
  7. TDM Data Grouping
  8. TDM Custom Properties
  9. DataPlugins
  10. Conclusion

1. Introduction

To reduce the need to design and maintain your own data file format, National Instruments has created a flexible file format called TDM, which is natively accessible through NI LabVIEW, LabWindows/CVI, and DIAdem, and is portable to other applications such as Excel. The TDM file format offers several unique benefits such as the ability to scale to your specific project requirements and to easily attach descriptive information to your measurements.

Read this paper to learn more about the TDM data model and see how LabVIEW data storage VIs reduce your programming time while increasing the value of the data you are collecting.


Overview of the Data Model

The TDM file format specifies that descriptive information – comments, units, and channel names – is saved in a header file with the extension TDM, while the bulk measurement, simulation, and analysis results are saved in a bulk binary data file with the extension TDX. The TDM and TDX files are linked through references automatically created with the LabVIEW data storage VIs.

Using XML, TDM files manage and structure all the descriptive information contained in the TDM header file. Each TDM file contains descriptive information on three different levels – root, group, and channel – as shown in Figure 1.



Figure 1. Each TDM file, written in NI LabVIEW, contains descriptive information on the root, group, and channel levels.


One TDM root object, also referred to as a file, can contain several groups, and each group can contain several channels. You can work with the TDM data model-defined set of properties at each of the three levels or insert your own custom properties.

The descriptive information located in the TDM header file, a key benefit of this format, provides an easy way to document the data without having to design your own header structure. As your saved-information requirements increase, you do not have to redesign your application; you simply extend the TDM data model to meet your specific needs.

Back to Top

2. LabVIEW Data Storage VIs for Writing, Reading, and Querying TDM Files


As with other file I/O functions in LabVIEW, you have to open, write to, and close the file in data storage VIs. But you can use the data storage VI configuration nature to save the properties you choose rather then worrying about file formatting, data types, and other tasks normally associated with writing additional information into a measurement file. The following sections step you through the basics of writing and reading, creating custom properties, and sharing your TDM files.

Back to Top

3. TDM Writing Data Basics


The simplest form of writing measurement data with the LabVIEW data storage VIs is illustrated below, where a data acquisition task generates an array of measurement values to the Write Data VI.



Figure 2. Data storage VIs document data at the channel level.


Notice that the Write Data VI has two inputs – Name and Unit – in addition to the Signal input. You choose which additional information you wish to save through a configuration dialog that you access by double-clicking on the Write Data VI. In the above example, all the channels coming from DAQ Assistant are given the same name and unit, however, the data storage VIs enumerate the channel name to prevent naming collisions. You can override the autoenumeration by passing an array of channel names to the Name input or by using an individual Write Data VI for each channel, both of which are acceptable solutions. An additional benefit of the Write Data VI is that it automatically calculates and saves the minimum and maximum values for each measurement array and saves this in the TDM file.

Figure 2 outlines how the data storage VIs document data at the channel level. Figure 3 illustrates how you can use the Set Properties VI to save additional descriptive information at the file level.



Figure 3. Use the Set Properties VI to save additional descriptive information at the file level.

 

Back to Top

4. TDM Reading Basics

Figure 4 illustrates the simplest form of reading data using data storage VIs. Here, the Open Storage VI passes a file reference to the Read Data VI and then all data is read from the TDM file into memory and displayed on a waveform graph.



Figure 4. Open Storage VI passes a file reference to the Read Data VI and then all data is read from the TDM file into memory and displayed on a waveform graph.


Querying data based on a channel property is unique to LabVIEW data storage VIs and saves programming time in applications that need to selectively load channels from a data set.

Back to Top

5. TDM Reading with Query Conditions


An additional Read Data VI feature is the ability to query data from a TDM file. Figure 5 shows the same Read Data VI, however, it has been configured to read back only the channels that have a maximum value greater than 1.



Figure 5. The Read Data VI is configured to read back only the channels that have a maximum value greater than 1.


Querying data based on a channel property is unique to LabVIEW data storage VIs and saves programming time in applications that need to selectively load channels from a data set.

Back to Top

6. TDM Reading a Subset of a Data Channel


As Figure 5 illustrated, you can selectively load channels from a TDM file using the query conditions on the read block. Though Figure 5 demonstrated selective loading using the maximum value property of a data channel, you are free to choose any of the other channel properties – for example, name, unit, or description – to establish a condition.

Figure 6 shows how you can load a subset of a data channel using the Get Properties VI. Building on the previous example, this VI loads only channels that have a maximum value greater than 1. Notice, however, that the Read Data VI is missing the signal output from the previous example. In this case, the signal output was turned off in the configuration dialog, preventing all the measurement data from being loaded into memory. Rather then pass arrays of data, the Read Data VI passes an array of channel refnums to the Get Properties VI, which requires a for loop to process. Also notice that the Get Properties VI has two additional inputs – index and count inputs. With these inputs, you can set the number of data points to read back as well as the index to start from in the file. This is particularly useful if you are working with data files that contain lengthy measurement arrays, but you only want to load a subsection into memory at a time.



Figure 6. Load a subset of a data channel using the Get Properties VI.

 

Back to Top

7. TDM Data Grouping

Figure 7 demonstrate a way to organize the data into channel groups. Groups are useful not only to separate measurement data from analysis results, but also to group together similar sensors. For example, channel group 1 can contain all the pressure sensor data and group 2 all the accelerometer data. You can use the same Read Data VI shown above reading channel data to read group information.


Figure 7. All the channels with a maximum value greater than 1 are loading into memory and passing to the fast Fourier transform calculation.


Back to Top

8. TDM Custom Properties


The TDM data model has a base set of properties at the file, group, and channel levels that you can use. However, in many cases, you may want to create a property that is specific to your particular needs,such as UUT, serial number, and test temperature. Using the Configure Set Properties dialog, you can add any custom properties you would like to either the File, Channel Group, or Channel level.



Figure 8. Configure Set Properties Window



Simply double click on the Set Properties VI and select “Configure Now”. The Configure Set Properties window also allows you to automatically include DAQmx Channel Properties in your TDM file through the DAQmx Tab of the Configure Set Properties Window.


Figure 9 DAQmx Configure Set Properties Window

 

Back to Top

9. DataPlugins

In addition to the TDM data type, LabVIEW Storage VIs are compatible with any data type for which you have a DataPlugin. DataPlugins are VBS files that define the format of a given file extension. National Instruments already has a wide variety of DataPlugins available on our DataPlugins page. In order to use a DataPlugin with the Data Storage VIs in LabVIEW, simply download the Load_With_DataPlugin library from our website. Then simply specify which DataPlugin you wish to use on the block diagram, as show in Figure 10.


Figure 10 DataPlugin Load File.vi


Although there may not be a DataPlugin listed for every data type, you can easily make DataPlugins using DIAdem. To download the Load_With_DataPlugin library as well obtain additional information on creating DataPlugins from DIAdem and using DataPlugins in LabVIEW, see the DataPlugins Programmer’s Reference Guide and the example Loading Data with a DataPlugin Within LabVIEW.



Additional TDM Information

Now that you are using the LabVIEW data storage VIs to properly document your data, you can use new data mining technology in NI DIAdem software to find trends and correlations in yout data. The DIAdem DataFinder automatically aggregates all of the meta information for your data files in a searchable index offering a self-configuring data management system that requires no IT effort or substantial investment in database software. It provides you intuitive access, searching and correlation capabilities for all your data files regardless of format. More information on DIAdem and the DataFinder can be found here.

Please note that the LabVIEW data storage express VIs do not work on real-time systems and are not designed for high speed streaming applications. However in LabVIEW 8.20 and higher there is an additional TDM Streaming API available which also works natively on real-time systems. More information on using TDMS VIs can be found here. Alternatively, there is an additional set of VIs available for writing TDM header files for your existing binary data files. The TDM Header Writer VIs can be found here.

Back to Top

10. Conclusion


LabVIEW offers a variety of file I/O options for reading and writing data. The data storage VIs provide several benefits that go beyond traditional file operations in LabVIEW. These benefits include:

1. The ability to easily save descriptive information with test data, which makes it conducive for posttest data mining
2. Channel groups that provide structure to data (for example, Group 1 = all temperature sensors, Group 2 = all pressure sensors)
3. Easy-to-use API that abstracts the low-level file I/O normally associated with saving data
4. Useful built-in functions for querying specific pieces of information from a TDM file and selective loading capabilities

TDM files are natively supported in National Instruments LabVIEW, LabWindows/CVI, and DIAdem software. An add-in is also available to load TDM files into Microsoft Excel. Visit ni.com/diadem to download the Excel TDM add-in.

Back to Top

Bookmark & Share


Ratings

Rate this document

Answered Your Question?
Yes No

Submit