In the end, the goal of every acquisition system is to analyze and visualize data quickly to get to results. But even though the data collection bottlenecks have been resolved, you can still lose efficiency when handling the data for analysis and reporting. One major bottleneck is how best to store data to disk. The number of different formats is limitless yet none seems to truly match the needs of engineers collecting measurement data at high speeds. This is why National Instruments introduced the Technical Data Management Streaming (TDMS) file format for streaming measurement data. The TDMS file format is a well-organized and self-documenting format that is optimized for high-speed streaming applications. Using TDMS, you can store and group channels of data and properties within a file to allow for easier postprocessing and exchangeability.
Figure 4. Each TDMS file contains descriptive information on the file, group, and channel levels.
Also, TDMS has been optimized for high-speed streaming to remove the bottleneck that previously existed for saving data to disk. In the past, engineers worked around the limitations of the file format with approaches such as decimating the data being written to the file, but this resulted in the literal loss of data. With TDMS, you can reach write speeds of up to 400 MB/s, removing the bottleneck introduced by file I/O.
Also, for extremely high speeds and ease of use, you can use the Configure Logging VI installed with the NI-DAQmx driver. This VI integrates NI-DAQmx scaling information in the header of the TDMS file so that only the raw data is written to file. Using just this single VI, you can achieve file writing speeds of up to 1.2 GB/s.
Figure 5. The Configure Logging VI is the fastest and easiest way to stream data to disk in LabVIEW.
Once you have stored all of this data to disk, you need to turn the raw data into meaningful information. Typically, engineers try to use common analysis packages such as Excel to sift through the data to find results. However, these spreadsheet programs were not designed for engineers. They cannot handle the large amounts of data being collected, and they do not include engineering analysis or make it easy to find trends within the data.
NI software such as LabVIEW and DIAdem was designed to help engineers get to results quickly. This software handles the large amounts of data engineers collect and includes the analysis and reporting tools that engineers require. And now, with the integration of NI DataFinder technology, you can quickly find the trends and correlations in your data without being bogged down by the sheer amount of data collected.
Figure 6. Postprocessing tools designed for engineers such as NI DIAdem make it easy to quickly analyze and report engineering data.
NI DataFinder is a data index that stores properties associated with your test data files. Therefore, you can create queries in LabVIEW or DIAdem that search through your data to find trends and outliers in your test data based on these properties. Once you have pinpointed this data, you can quickly process, visualize, and report on it using LabVIEW or DIAdem. Using NI DataFinder, you can quickly find the data that matters most and get to results. For more information on NI DataFinder data management technology, visit Best Practices for Saving Measurement Data.
Figure 7. Using NI DataFinder in LabVIEW, you can create applications to quickly find trends and outliers and generate results faster.