PDF1. Introduction
Creating TDM files in LabVIEW is very simple with the help of the Data Storage VIs. In certain cases the Data Storage VIs may not be able to meet the application requirements. If your application requires high-speed streaming of data, the Data Storage VIs might not perform at the required speed. If you are using a real-time system, the Data Storage VIs are not available. You might also have existing binary files that you want to easily convert to the TDM format. For these reasons National Instruments has developed LabVIEW TDM Header Writer VIs. With this easy-to-use interface, you can take advantage of all of the TDM file functionality with binary files created during high-speed data acquisition, data acquisition in a real-time system, or existing binary files.
The remainder of this paper will discuss the use of the TDM Header Writer VIs to create TDM header files for existing binary files The TDM Header Writer VIs referenced in this paper are available for download (see link at the bottom of this application note).
2. Overview of the TDM Data Model
The TDM file format specifies that descriptive information such as comments, units, and channel names are saved in a header file with the extension TDM, while the measurement, simulation, or analysis results are saved in one or more binary file(s). The TDM and binary files are linked through references that are created automatically when using the LabVIEW Data Storage VIs, or programmatically when using the TDM Header Writer VIs.
TDM files use XML to manage and structure all the descriptive information contained in the TDM header file. Each TDM file contains descriptive information on three different levels – root, group, and channel – as illustrated below.
One TDM root object, also referred to as file, can contain several groups, and each group can contain several channels. The TDM data model has a defined set of properties at each of the three levels while also providing functionality for you to insert your own custom properties.
The descriptive information located in the TDM header file is a key benefit of this format because it provides an easy way to document your data, without having to design a custom header structure. As your requirements for information saved with each measurement grow, you do not have to redesign your application; you simply extend the TDM data model to meet your expanding needs.
The division between header information and measurement data makes the TDM files very flexible, giving you the opportunity to acquire data using multiple methods capable of storing binary data and later document the information with a TDM header file. The TDM Header Writer VIs provide an easy-to-use interface to create the TDM header file.
It is common for standard LabVIEW binary files to contain measurement information from different data channels acquired at different rates and stored with different data types. The file can store data in signed or unsigned integers ranging from one to eight bytes in length, and single and double-precision floating-point numbers. (Note: Although the TDM header file can describe the previously mentioned data types, LabVIEW versions prior to 8 and DIAdem versions prior to 10.1 will not read 64-bit integers and 8-bit signed integers). The binary data can be stored in big or little endian format. The TDM header file can accommodate mixed data rates and data types, as you will see in the following examples.
3. Creating a TDM Header File for a Single-Channel Binary File
One of the characteristics of TDM files is that they are capable of storing both very simple and complex data sets. One of the simplest binary files used to create a TDM header file is a single-channel binary file. This file consists of a single block of binary data. A block is a set of contiguous data points describing the value of one or more channels. The easiest way of creating such a file is to use the Write Binary File VI that is shipped as an example program with LabVIEW. Running this example will generate a binary file with 10 double-precision floating-point numbers.
The first three steps of creating a TDM header for the single-channel binary file are shown below.
The first step in creating a TDM header file is to create the file itself. The Create TDM File VI will create the TDM file with a name defined by TDM Header Name input and a path defined by the TDM Header File Path input. As an output this VI returns a reference to the newly created TDM file.
Using the reference to the TDM file, the Setup Binary File VI links the binary file containing the data we want to describe with the TDM header file. This VI receives as inputs the name and path of the binary file containing the data and the byte order of the file. TDM files handle binary files stored both in big and little endian byte order. In the case of the binary file created by the Write Binary File example VI, the file is stored in big endian byte order.
Now that we have linked the binary data to the TDM file, we must specify the format of the binary data. The Setup Channel VI defines the format of the binary file by accepting the byte offset, length and data type of the file. The byte offset specifies the number of bytes at the beginning of the file that must be ignored before reaching the measurement data. This functionality facilitates the use of existing binary files by enabling you to ignore headers written at the beginning of the file. The length input specifies the number of samples stored in the file. Finally, the data type determines in which format the data was stored for this particular channel. In the case of the Write Binary File example VI, the data was stored as a double-precision floating-point number. The Setup Channel VI returns a Binary Channel Index that will be used later on when we define the properties for the channel.
After defining the format of the binary file we can create the descriptive structure of the TDM file. This process is illustrated below.
Using the reference for the TDM header file, the Create Root VI creates the Root object of the file and defines its properties. Some of the properties defined at the Root object include; Name, Description, Title, Author and Number of Groups. Of all of these properties Number of Groups is the only one that will affect the structure of the TDM file.
The next level of the TDM file structure is the Groups. Because we only have one Group in the above example, we will use one instance of the Create Group VI. If we had more than one Group, other instances of this VI would be necessary. The three properties for the Group we created are its name, description and number of Channels.
The last step in creating the TDM header file is to specify the properties for Channels. This operation is performed by the Create Channel VI with the Binary Channel Index as an input. The Binary Channel Index is created previously when we defined the format of the binary file using the Setup Channel VI. At this point we can set the channel properties such as name, description, and units. Finally, to complete the creation of the TDM file, we must close the file using the Close TDM File VI.
4. Creating a TDM Header File for a Multichannel Binary File
One of the great advantages of TDM header files is that they can describe many different types of data sets. The most common use-case for TDM header files is documenting binary data acquired from multiple channels.
In order to link data from multiple channels in a single binary file to a TDM header, the channel data must be stored either end-to-end or interleaved. The most common case is data from multiple channels stored in a single, interleaved binary block. This binary block will not only be described by its length, the number of samples per channel, but also by its width, the number of bytes containing one sample for all channels. Illustrated below is an example of the structure of a binary file containing four channels and n samples.
In this case the binary block length is n. If the data type of all of the channels was an 8-byte double-precision floating-point, the block width would be 32 bytes. Note that the channels in the same binary block may be stored with differing data types. Note also that the channels must be consistently interleaved in consecutive order.
One of the most common ways of acquiring data from multiple channels in LabVIEW is to use the NI-DAQmx Read VI. If the output of the NI-DAQmx Read VI is stored in a binary file, it is formatted end-to-end. Although a TDM header file can describe data stored in end-to-end format using the Setup Binary File VI, the following example assumes that the data is interleaved. To transpose the output of the NI-DAQmx Read VI from end-to-end to interleaved format, use the Transpose 2D Array VI. The following example demonstrates how to acquire 1000 samples in each of four channels, transpose the data from end-to-end to interleaved format, and save the information to a binary file in interleaved format. (Note: TDM header files describing binary files in interleaved format can only be read in LabVIEW versions 8 or later and in DIAdem versions 9.1 SP2 or later.).
The process of creating a TDM header file for a binary file containing multiple channels is very similar to creating one for a binary file a single channel. The first of two differences is that instead of using a single instance of the Setup Channel VI to define the format of the binary file, we will use one instance of the Setup Block Channel VI for each of the channels in the file. The second difference is that instead of using one instance of the Create Channel VI to define the properties of a single channel, we will use one instance of this VI for each of the channels in the file.
The first two steps in creating the TDM header file for a binary file with multiple channels will be the same as those used to create a TDM header file for a binary file with a single channel, as the following block diagram illustrates.
The third step of the process involves using the Setup Block Channel VI instead of the Setup Channel VI. The Setup Block Channel VI is used because with it we can specify the size of a block in the binary file as well as the relative position of the channels in the block. The VI is called once for each of channels in the file; therefore it is in a For loop structure that iterates once for each channel. The first input of the Setup Block Channel VI is the block offset. This input determines the number of bytes from the beginning of the file where the first block of data is located. The next input is the block width in bytes. The third input determines the file block length.
All of the previous inputs will have the same values for each of the channels in our file. The third input, Channel Byte Offset, will be different for each channel. This input specifies the number of bytes from the beginning of the file to the first sample of a certain channel. The last input is an enumerator that determines the data type in which the samples were stored. The Setup Block Channel VI outputs a Block Channel Index for each of the channels whose format we specified. These four Block Channel Indices are added to an array that will later be used by the Create Channel VI in order to set these channels’ properties.
After creating the TDM header file, linking the binary file, and defining its format, we are ready to set the properties for the TDM header hierarchy. This process is exemplified below.
There is very little difference between the last four steps of creating a TDM header file for a multichannel and a single-channel binary file. The TDM file only has one root object so this is created in the same way, and because we created only one group this operation is identical except that the Number of Channels input is now equal to 4. The main difference is that the Create Channel VI is called once for each channel in the binary file. The Block Channel Indices coming from the Setup Block Channel VI determine the number of iterations of the For loop structure and are accepted as a parameter in the Create Channel VI.
5. Creating a TDM Header File for Multiple Binary Files
One of the biggest drawbacks of using binary files is that any documentation or header information is limited to the file where it resides. TDM header files are much more flexible, so you can document and organize different binary files into a cohesive structure. These binary files can even have different samples counts, byte order, formatting, and data types. The following example demonstrates how to create a TDM header file for two different binary files. One file contains a block for binary data for a single channel of data in double-precision floating-point format and the second a block containing a single channel of data in 32-bit integer format. The following block diagram demonstrates how to link the binary files to the TDM file and define their format.
The process of building a TDM header file for multiple binary files follows the same structure as creating one for a single binary file. First, the TDM header is created using the Create TDM File VI. Then the Setup Binary File VI is used to link the first binary file to the TDM header. After linking the binary file we define its format using the Setup Binary File VI that will return a Binary Channel Index that is used later by the Create Channel VI. The big difference when building a TDM header file for two binary files is that the Setup Binary File and Setup Channel VIs must be called twice, once for each file. It is necessary that the Setup Binary File VI is called immediately before the Setup Channel VI for each file. The two previously mentioned VIs must also be called consecutively for each of the files.
In order to assign properties to the binary file in the TDM header file, we will use the same three VIs used in the previous two examples. The process is the same as assigning properties for a single file, except that in this case we will be creating two groups, one for each binary file, instead of one. We will call the Create Group followed by the Create Channel VIs consecutively for each binary file. It is required that these two VIs are called in that order; otherwise the TDM header file is not created correctly. The Binary Channel Indices created by the two Setup Channel VIs are used by their respective Create Channel. Finally, the TDM header is closed by the Close TDM File VI.
6. Creating a TDM Header File and Setting Properties
Using TDM files you can specify predefined and custom properties for your data at different hierarchical levels. This functionality makes TDM files excellent for documenting large data sets with expanding documentation needs. Using the TDM header writer VIs, you can assign and create properties programmatically as the following block diagram illustrates.
The first two steps of assigning properties to a TDM file are to create a TDM file and a Root element. To set an existing property, use the Add Base Model Property VI. This VI accepts two inputs, the name of the property and its value. This VI can be used to set properties at either the Root, Group, or Channel level. The Add Base Model Property VI must follow the VI creating the hierarchical level that the property will be describing. The next step in the sequence involves creating a Group. The Add Custom Property VI is used to create and assign a property to the group. The two inputs used by the VI are the name of the property to be created and its value. Like the Add Base Model VI, the Add Custom Property VI must be called following the creation of the hierarchical level to which the property will be assigned. The last stage of the process closes the TDM file.
7. Conclusion
The TDM Header Writer VIs provide an easy-to-use interface for creating TDM header files for existing binary files. This flexibility provides TDM files with several benefits over other file formats, they include:
1. Stream data very quickly to a binary file and then create a TDM header file to document the properties and structure of the data
2. Describe the properties and structure of data in existing binary files by creating a TDM header file
3. Create TDM files out of binary files acquired in real-time systems
4. Consolidate data and documentation from multiple binary files into a single TDM file
5. Use existing APIs such as LabVIEW Data Storage VIs
8. Appendix: TDM Header Example Description
Example Create Block Channels.vi: Creates a TDM header file for a binary file containing four channels with 2000 samples of integers
Example Create Group.vi: Creates a TDM header file containing a single Group
Example Create Implicit Channel.vi: Creates an implicit channel in a TDM header file
Example Create Normal Channel.vi: Creates a TDM header file for a binary file containing a single channel with 2000 samples of integers
Example Create Waveform Channel.vi: Creates a waveform channel in a TDM header file for a binary file containing a single channel with 2000 samples of integers
Example Multiple Binary Files.vi: Creates a TDM header file two binary files, the first of which contains a single channel of 2000 integer samples; the second contains three channels with 2000 samples of integers
Example Set Properties.vi: Sets the value of an existing property to the Root of a TDM file, and creates and assigns the value of the Group property
Related Links:
Example Code: TDM Header Writter VIs
Application Note: Introduction to the LabVIEW Data Storage VIs
