Print
PDFOverview
Why do you collect data? Is it to validate the safety of your product? Do you need to find trends over long periods of time to make knowledge-driven decisions? Chances are, you are facing a challenge, and you are running tests and using the data to find a solution or determine a next step. The data generated from these tests is valuable. Often, you have only one chance to run a test—and as test complexity and frequency increases, so does the cost. When developing your testing system, you need to answer a few questions:
- Are you using the correct file format to collect data?
- Can the file format adapt as your project needs change?
- Do you have a way to easily find the data you need?
- Are you using the right tools to extract knowledge from your raw data?
NI recognizes the challenges associated with storing, managing, and making sense of the data collected during tests. Technical Data Management (TDM) technologies, including the TDM Streaming (TDMS) file format for data storage, NI DataFinder for data indexing, and NI LabVIEW and NI DIAdem software for data processing, directly integrate into your applications to help you maximize your test and measurement data value. Every year, NI makes improvements to TDM technologies. Read on to find out more about some new features NI has included in the 2012 product releases.
Table of Contents
- Added Support for TDMS on Mac and Linux
- TDMS Asynchronous I/O Support for LabVIEW External Data Value Reference
- TDMS Defragment Indicator
- New NI DataFinder Features
- New DIAdem Features
- Analyze More Data With New Functions
- Develop Scripts for Faster Reporting
- Check for Updates to DataPlugins From Within DIAdem
- Additional Resources
National Instruments developed the TDMS file format specifically to meet the needs of engineers and scientists who collect test data. TDMS is a binary-based file format, so it has a small disk footprint and can stream data to disk at high speeds. Even though the TDMS files are binary, you can open them in many common applications, such as Microsoft Excel and OpenOffice, for sharing with colleagues. TDMS files also contain a header component that stores descriptive information, or attributes, with the data. Thus, TDMS files give you the benefits of easy exchangeability and attribute inclusion without sacrificing speed and size. NI also has added numerous performance and usability features so that you can write data to file in the most efficient manner possible.
Added Support for TDMS on Mac and Linux
NI listened to your feedback, and with LabVIEW 2012, is introducing built-in support for the TDMS API on Mac and Linux. The TDMS standard and advanced APIs are available for the three new supported platforms.
TDMS Asynchronous I/O Support for LabVIEW External Data Value Reference
NI has worked hard to make streaming your data easier and with as high performance as possible. With the new LabVIEW External Data Value Reference (EDVR), you can directly access a portion of a driver’s DMA buffer. This means that you can log data from reconfigurable I/O (RIO)-based hardware directly to disk without any memory copies, and can reach an I/O throughput of 3.2 GB/s. The new Asynchronous I/O palettes vastly improve total memory bandwidth and CPU utilization, leaving your RIO-based hardware available to process more data.
| Metric | Before (With Copies) | After (Zero-Copy) |
| CPU Utilization | 74% | 11% |
| Total Memory Bandwidth | 11.2 GB/s | 6.2 GB/s |
Table 1. CPU Utilization and Total Memory Bandwidth Improvements for RIO-Based Hardware Using the Asynchronous I/O TDMS Palettes when using NI-RIO 2011 and NI-RIO 2012
TDMS Defragment Indicator
As data is written to different channels in a TDMS file, a header is written along with the data. As you collect data for long periods of time, the file size can grow and affect the performance of clients such as LabVIEW, DIAdem, and NI DataFinder. To ensure that your data applications are always running at top speed, the TDMS Defragment Indicator suggests instances in which NI recommends that you clean up your TDMS files to increase performance.
Figure 1. A new property is included in NI DataFinder to indicate if a file needs to be defragmented.
New NI DataFinder Features
NI DataFinder is, without a doubt, the cornerstone of the NI TDM platform. It is an indexing service that parses any file format for descriptive information (properties) and creates a database of the descriptive information within your data files. This database is automatically updated as soon as a valid data file is created, deleted, or edited. With NI DataFinder, you can easily search test data stored within your local index; however, needs and requirements change as you expand a data management solution to large groups or departments. To meet these needs, National Instruments developed NI DataFinder Server Edition. The latest release focuses on improving NI DataFinder Server Edition performance to quickly and efficiently parse large data file sets.
Table 2. Decrease the time spent indexing large data sets with NI DataFinder Server Edition 2012.
New DIAdem Features
DIAdem software is specifically designed to make you more productive and more efficient when visualizing, analyzing, and reporting data. This frees you up to spend more time on the tasks that truly matter—unlocking information from data to make decisions. From its comprehensive engineering analysis functions to its large data set processing, DIAdem has always treated measurement data as a first-class citizen. DIAdem 2012 introduces several new features that make data processing in DIAdem even faster and easier than before.
Analyze More Data With New Functions
You can analyze your data using new engineering functions, which range from basic math to signal processing and more. When you use a built-in DIAdem analysis function, you can immediately preview the results in the configuration dialog box and interactively change calculation parameters until you find the best fit for your data.
Figure 2. DIAdem offers a wide range of powerful, mouse-driven mathematical routines for analyzing your data.
Develop Scripts for Faster Reporting
Reduce your programming time and effort using the new object-oriented VBScript API for reporting. It features syntax highlighting and code completion. You also have the option to create your report using the interactive drag-and-drop editor and modify your custom reports to meet your automation needs. All DIAdem reports are saved as templates, so you can design a report once, using either method, and reuse it with any data set.
Check for Updates to DataPlugins From Within DIAdem
With exclusive DataPlugin technology, DIAdem can read data saved to any file extension or custom file format. You can download more than 400 DataPlugins from ni.com/dataplugins or write your own if your file is proprietary—or the DIAdem team can write one for you. Once you are in the DIAdem environment, ensure you always have the most current DataPlugins with the DataPlugin Update Manager, which checks ni.com/dataplugins for updates.
Additional Resources
To lean more about the TDMS file format, visit ni.com/tdms
To get more information about how NI DataFinder can help you manage your data in your department, visit ni.com/datafinder
To view a webcast demonstrating DIAdem 2012 features, visit http://zone.ni.com/wv/app/doc/p/id/wv-1392.
Reader Comments | Submit a comment »
Legal
This tutorial (this "tutorial") was developed by National Instruments ("NI"). Although technical support of this tutorial may be made available by National Instruments, the content in this tutorial may not be completely tested and verified, and NI does not guarantee its quality in any way or that NI will continue to support this content with each new revision of related products and drivers. THIS TUTORIAL IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND AND SUBJECT TO CERTAIN RESTRICTIONS AS MORE SPECIFICALLY SET FORTH IN NI.COM'S TERMS OF USE (http://ni.com/legal/termsofuse/unitedstates/us/).