Introduction to the Measurement Studio TDM Streaming .NET Class Library

Publish Date: Aug 03, 2012 | 2 Ratings | 5.00 out of 5 |  PDF

Overview

This white paper provides an overview of features and use cases for the National Instruments Measurement Studio TDMS .NET Class Library.

Table of Contents

  1. Introduction
  2. The TDMS File Format of the TDM Data Model
  3. Measurement Studio TDM Streaming API for Writing and Reading TDMS Files
  4. Additional TDMS Information
  5. Conclusion
  6. Next Steps

1. Introduction

To reduce the need to design and maintain your own data file format, National Instruments has created a flexible data model called technical data management (TDM), which is natively accessible through NI Measurement Studio, LabWindows/CVI, LabVIEW and DIAdem and is portable to other applications such as Microsoft Excel. A TDM DLL is also available for accessing the TDM data model with third-party software applications. The TDM data model offers several unique benefits, including the abilities to scale to your specific project requirements and to easily attach descriptive information to your measurements while streaming your data to disk. The TDM data model supports two file formats, TDM and TDM Streaming (TDMS). The new TDMS .NET Class Library is optimized for streaming well-documented data to disk. The other file format of the TDM data model, written by the TDM DLL, offers the same flexibility with regard to documenting data, but is not optimized for streaming data to disk. The new TDMS API provides the widest coverage of different use cases, and as a result it is the recommended way to interface with the TDM data model. The TDM white paper provides additional information including a comprehensive overview of the NI TDM platform.

This white paper explains the TDM data model and demonstrates how the Measurement Studio TDMS API reduces your programming time while increasing the value of the data you are collecting.

Back to Top

2. The TDMS File Format of the TDM Data Model

The TDM data model offers three levels of hierarchy – root, group, and channel (see Figure 1). Each level accepts an unlimited number of user-defined attributes, making the data “search ready.”


Figure 1. Each TDMS file written in Measurement Studio contains descriptive information on the root, group, and channel levels.

Unlike TDM files, which have a strictly required XML-based header file, TDMS files have a binary index file with the extension *.TDMS_Index. The TDMS_Index file provides consolidated information on all the attributes and pointers in the bulk data file, and accelerates access to the data while reading. The TDMS_Index file is automatically regenerated if deleted or removed.

 

The TDMS file can contain several groups, and each group can contain several channels. You can insert your own custom properties at each of the three levels. Creating and using channel groups and custom properties are discussed in greater depth later in this white paper.

 

The descriptive information located in the TDMS file, a key benefit of this format, provides an easy way to document the data without having to design your own header structure. As your documentation requirements increase, you do not have to redesign your application; you simply extend the TDMS data model to meet your specific needs.

Back to Top

3. Measurement Studio TDM Streaming API for Writing and Reading TDMS Files

Like other file I/O functions in Measurement Studio and .NET, you have to open, write to, and close the file with the TDMS API. You can also use the native TDMS functions from the API to save the properties you choose rather than worrying about file formatting, data types, and other tasks normally associated with writing additional information to a measurement file. Review the following sections for the basic steps of writing, reading, creating properties, and sharing your TDMS files.

TDMS Writing Data Basics

The simplest form of writing measurement data with the Measurement Studio TDMS functions is illustrated in Figure 2. In this example, a sine wave is generated, saved as an array of measurement values, and passed to the TdmsChannel.AppendData function.

 

using System;
using System.Collections.Generic;
using System.Text;
using NationalInstruments.Tdms;
using NationalInstruments.Analysis.SignalGeneration;

namespace WriteDataBasic
{

class Program
{

static void Main(string[] args)
{

double []measuredData;
TdmsFile file;
TdmsChannelGroup mainChanGroup;
TdmsChannel mainChannel;

//Create and set up TDMS file
file = new TdmsFile("C:\\signals.tdms", TdmsFileAccess.ReadWrite);
file.Description = "DAQ Test";
mainChanGroup = file.AddChannelGroup("Main Group", "Dev1");
mainChannel = mainChanGroup.AddChannel("Main Channel", TdmsDataType.Double, "ai0");

//Write data
SineSignal signal = new SineSignal();
measuredData = signal.Generate(100, 200);
mainChannel.AppendData(measuredData);

//Close file
file.Close();

}

}

}

 
Figure 2. Write data to a TDMS file at the channel level.

Notice the calls to the TdmsFile.AddChannelGroup function and the TdmsChannelGroup.AddChannel function. In the example above, the channel associated with the sine wave data is described by a specific group and channel name. The acquired signal is then appended to the channel identified in the TdmsChannelGroup.AddChannel function. Figure 3 demonstrates the structure of the TDMS file written in the example code above.

Figure 3. TDMS files store descriptive information at the file, group, and channel levels.

TDMS Reading Data Basics

Figure 4 illustrates the simplest form of reading data using the TDMS API functions. Here, the TdmsFile object obtains a file reference to the TDMS file and calls the TdmsFile.GetChannelGroups function. After getting the appropriate channel from the channel group and the number of values stored in that channel, the TdmsChannel.GetData function reads all data in “Main Group” into memory.

 

using System;
using System.Collections.Generic;
using System.Text;
using NationalInstruments.Tdms;

namespace ReadDataBasic
{

class Program
{

static void Main(string[] args)
{

long numChanValues;
double []measuredData;
TdmsFile file;
TdmsChannelGroupCollection channelGroups;
TdmsChannelCollection channels;

//Open TDMS file
file = new TdmsFile("C:\\signals.tdms", TdmsFileAccess.Read);

//Read group data
channelGroups = file.GetChannelGroups();
channels = channelGroups[0].GetChannels();
numChanValues = channels[0].DataCount;
measuredData = channels[0].GetData<double>();

//Close file
file.Close();

}

}

}


Figure 4. Read data back from a TDMS file.

TDMS Write Custom Properties


The TDMS data model automatically creates certain properties when you write some data types. However, in many cases, you may want to create a property specific to your particular needs, such as UUT, serial number, or test temperature. You can accomplish this task using the AddProperty function, which you can use to write properties at the file, group, or channel levels, as shown in Figure 5.

 

The file level of a property determines which set of property functions need to be called. For example, to write a group property, only the TdmsChannelGroup.AddProperty function needs to be called. This is also true when setting file and channel properties. These cases are illustrated in Figure 5.

 

using System;
using System.Collections.Generic;
using System.Text;
using NationalInstruments.Analysis.SignalGeneration;
using NationalInstruments.Tdms;

namespace WriteCustomPropertiesDifferentLevels
{

class Program
{

static void Main(string[] args)
{

double[] measuredData;
TdmsFile file;
TdmsChannelGroup mainChanGroup;
TdmsChannel mainChannel;

string[] filePropertyNames = { "TestID", "Plant" };
string[] filePropertyValues = { "Test1", "Site B" };
string groupPropertyName = "Serial #";
string groupPropertyValue = "012345";
string[] chanPropertyNames = { "Max Temp", "Min Temp" };
int[] chanPropertyValues = { 100, 0 };

//Create and set up TDMS file
file = new TdmsFile("C:\\signals.tdms", TdmsFileAccess.ReadWrite);
file.Description = "DAQ Test";
mainChanGroup = file.AddChannelGroup("Main Group", "Dev1");
mainChannel = mainChanGroup.AddChannel("Main Channel", TdmsDataType.Double, "ai0");

//Write file properties
for (int i = 0; i < 2; i++)

file.AddProperty(filePropertyNames[i], TdmsPropertyDataType.String, filePropertyValues[i]);

//Write group properties
mainChanGroup.AddProperty(groupPropertyName, TdmsPropertyDataType.String, groupPropertyValue);

//Write channel properties
for (int i = 0; i < 2; i++)

mainChannel.AddProperty(chanPropertyNames[i], TdmsPropertyDataType.Int32, chanPropertyValues[i]);

//Write data
SineSignal signal = new SineSignal();
measuredData = signal.Generate(100, 200);
mainChannel.AppendData(measuredData);

//Close file
file.Close();

}

}

}

Figure 5. Write custom properties at three different levels before writing data to the file.

With the TDMS AddProperty functions, you can specify an individual property by itself or specify many properties by using arrays. Figure 5 specifies two properties at the file level (TestID and Plant). You can expand this to specify many more properties by just increasing the size of the array. Arrays are not necessarily needed if only a single property is written, such as a serial number.

 

Property values can also be different data types. In Figure 5, string properties are written at the file and group levels. But at the channel level, two numeric properties are written to specify the minimum and maximum temperature.

 

Figure 6 illustrates how you can use the TDMS AddProperty function to save additional descriptive information at the file level. In this example, the date and the user of the test are built into an array and then written as custom properties.

 

using System;
using System.Collections.Generic;
using System.Text;
using NationalInstruments.Analysis.SignalGeneration;
using NationalInstruments.Tdms;

namespace WriteCustomPropertiesFileLevel
{

class Program
{

static void Main(string[] args)
{

double[] measuredData;
TdmsFile file;
TdmsChannelGroup mainChanGroup;
TdmsChannel mainChannel;

string[] customFilePropertyNames = { "Date", "Author" };
string[] customFilePropertyValues = { "", "" };

//Create and set up TDMS file
file = new TdmsFile("C:\\signals.tdms", TdmsFileAccess.ReadWrite);
file.Description = "DAQ Test";
mainChanGroup = file.AddChannelGroup("Main Group", "Dev1");
mainChannel = mainChanGroup.AddChannel("Main Channel", TdmsDataType.Double, "ai0");

customFilePropertyValues[0] = DateTime.Today.ToShortDateString();
customFilePropertyValues[1] = Environment.UserName;

//Write file properties
for (int i = 0; i < 2; i++)

file.AddProperty(customFilePropertyNames[i], TdmsPropertyDataType.String, customFilePropertyValues[i]);

//Write data
SineSignal signal = new SineSignal();
measuredData = signal.Generate(100, 200);
mainChannel.AppendData(measuredData);

//Close file
file.Close();

}

}

}

Figure 6. Use the AddProperty function to save additional descriptive information at the file level.

TDMS Read Custom Properties


Once you have written a TDMS file, you can read the properties back into Measurement Studio using the TDMS GetProperties functions. For example, Figure 7 shows how to get properties for the file level by using the TdmsFile.GetProperties function. This process is very similar to writing the properties.

 

using System;
using System.Collections.Generic;
using System.Text;
using NationalInstruments.Tdms;

namespace ReadTDMSProperties
{

class Program
{

static void Main(string[] args)
{

int numProperties;
TdmsFile file;
TdmsChannelGroupCollection channelGroups;
TdmsChannelCollection channels;
string[] allFilePropertyNames;
string[] allFilePropertyValues;

//Open TDMS file
file = new TdmsFile("C:\\signals.tdms", TdmsFileAccess.Read);

//Read file properties
channelGroups = file.GetChannelGroups();
channels = channelGroups[0].GetChannels();
numProperties = file.GetProperties().Count;
allFilePropertyNames = new string[numProperties];
allFilePropertyValues = new string[numProperties];
for (int i = 0; i < numProperties; i++)
{

allFilePropertyNames[i] = file.GetProperties()[i].Name;
allFilePropertyValues[i] = file.GetProperties()[i].GetValue<string>();

}

//Close file
file.Close();

}

}

}

Figure 7. Read TDMS properties from three levels in the data storage model.

In this configuration, the property values are returned as a string array because we specify the data type in the TdmsProperty.GetValue function. The data could be a string, double, boolean, or some other data type and the function would return an object array if you did not specify a data type. You can display the data on the user interface as an object or typecast it in .NET code to the appropriate data type.

 

TDMS Defragment File and TDMS Save File

Included with the TDMS API functions are the defragment and save functions for optimizing read and write operations.

Each call of the TDMS write functions records a block of data to file, which may contain several channels. This prevents the data for a single channel from being written contiguously across multiple writes. The TdmsFile.Defragment function reorganizes the file by channel to optimize read operations. In this way, the user can take the performance penalty only once by defragmenting the TDMS file “offline” rather than with every read. For example, the user is able to acquire and store data very rapidly and then defragment the file before performing any read operations.

The TdmsFile.Save function forces all cached data to disk, a useful feature to ensure that no data is lost in the event of a system crash. Using the TdmsFile.Save function costs some system performance. If AutoSave is true, the TdmsFile is automatically saved to disk after every change. If AutoSave is false, you must call Save explicitly to write the cached data to disk. AutoSave is set to true on the write functions by default so there is typically no need to manually call the Save function.

 

View TDMS Files

TDMS File Viewer Shipping Example

Measurement Studio is shipped with two well-documented examples demonstrating how to write and read TDMS files. The TDM Streaming example sets up a TDMS file and then streams data to disk. 

Figure 8. The TDM Streaming example documents the streaming capabilities of the TDMS file format.

The TDMS File Reader example loads the file generated by the TDM Streaming example, displays the different levels of data in the file such as groups and channels, and displays the properties and data inside each level that is selected from the tree structure. Figure 8 shows an example of the File Reader.

   

Figure 9. The TDMS File Reader provides easy access to your TDMS data file.

Load TDMS Files into Excel

Using the Microsoft Excel add-in, which is included with Measurement Studio 8.6 and later, you can load and work with  TDMS files in Excel. Excel limits the amount of data that you can load per data channel to 65,535 values (1,048,576 values in Excel 2007); however, with the Excel add-in, you can specify which starting point to read from if your measurements exceed this limit.

TDM Excel Add-In Tool for Microsoft Excel User Guide

Load TDMS Files into DIAdem

TDMS files can also be loaded and viewed in National Instrument DIAdem data management software. With the National Instruments DataFinder included in DIAdem, not only can you locate the data you need to work with faster, you also have the ability to mine your test data to search for trends and correlations.

In addition to being able to manage and mine your data in DIAdem, you can also perform perform advanced offline analysis on your data as well as generate automated reports.

Back to Top

4. Additional TDMS Information

TDMS files are native in National Instruments Measurement Studio, LabVIEW, LabWindows/CVI and DIAdem software.

Read More about the TDMS File Format Internal Structure 

Back to Top

5. Conclusion

Measurement Studio and .NET offer a variety of file I/O options for reading and writing data. The TDMS .NET Class Library provides benefits that go beyond traditional file operations in Measurement Studio. With the TDMS API, you can:

  1. Quickly stream data to disk  in a structured format.
  2. Easily save descriptive information with test data, making it search-ready.
  3. Categorize data under groups to add structure (for example, Group 1 = temperature sensors, Group 2 = pressure sensors, etc).
  4. Use the API to abstract the low-level file I/O normally associated with saving data.
  5. Include descriptive properties to help analyze data at a later time.

Back to Top

6. Next Steps

Run the Example Code in the Measurement Studio 7-Day Evaluation
View the Measurement Studio Resource Page

Back to Top

Bookmark & Share

Ratings

Rate this document

Answered Your Question?
Yes No

Submit