What is the NI DataFinder?


DataFinder technology is installed when you install DIAdem. DataFinder is an indexing service that parses any custom file format for descriptive information (properties) and creates a database of the descriptive information within your data files. This database is automatically updated as soon as a valid data file is created, deleted or edited.

You can think of the DataFinder as an iTunes for your data files. Both DataFinder and iTunes are management tools to help keep track of numerous files. iTunes generates a virtual library which keeps tracks of songs and their metadata (artist, song title, etc.) while DIAdem creates a database for all properties written to any of your data files. The iTunes virtual library will only interpret metadata from compatible file formats (MP3, WAV, etc), while DataFinder will only index properties from files that have a DataPlugin registered with DIAdem. The DataPlugin maps custom file formats to the TDMS model, which can have properties saved to the file, channel group, or channel levels.


Specifying Folders to Index

After confirming that you have iTunes or DataFinder configured to read in your files, you need to teach both programs where to look for these file. iTunes does not search every folder on your computer for music files. Instead iTunes looks only at directories that you specify from the Add Folder to Library… option or in a pre-designated iTunes Music folder. You can perform the same selective indexing of your data files with the DataFinder. You can add your files to one of three folders that by default are indexed by DIAdem (C:\Users\<username>\My Documents, C:\Users\Public\Documents\National Instruments, and C:\Program Files\National Instruments\DIAdem 2011\Examples). Or you can right-click on Search Areas and select Add Search Area…  as seen in Figure 1. Selective indexing prevents system resource overhead that results from unnecessarily indexing files or folders while allowing for easy scalability should you need to specify new search areas in the future. You also have the option to remove search areas by right-clicking on them and selecting Remove Search Area…

Figure 1 - Adding specific folders to index metadata in both iTunes and DIAdem

Searching for Files

Once you have customized your search areas and indexed the metadata, the next thing is to know is how to use this management tool to find the file that you need. Both iTunes and DataFinder have an Internet-like search bar that returns a list of filtered results based on the keywords you enter. With the help of DataPlugins, which map custom file formats onto the TDMS model, the DataFinder search looks at all of the metadata at the file, channel group and channel level. DIAdem allows you to perform advanced searches at each of these levels to find the exact data set that you need. You can even use common wildcards (*, ?) to search for results.

Figure 2 - Using an Internet-like search to find files in iTunes and performing an 
advanced search with wildcards in the DIAdem DataFinder

Creating a Custom Arrangements

When you listen to music, you may not want to listen to an album sequentially. iTunes allows you to create a playlist where you can choose what artist and what order you would like to listen to your music. Likewise when you are working with your data, you often want to compare results from multiple test runs. In the DIAdem Data Portal you can selectively load data at the file, channel group or channel levels to create your ‘data playlist’. You can then save a copy of this data playlist to reuse at another time without overwriting your original data files.

Figure 3 - A custom playlist in iTunes and a custom data set loaded into the Data Portal in DIAdem

Expanding Your Data Management System Beyond a Local Machine

The DIAdem DataFinder, without a doubt is the cornerstone of the NI data management solution and is designed for individuals and occasionally does not meet the needs of larger groups accessing data across multiple machines. As a results, National Instruments offers two data management options – DIAdem DataFinder and NI DataFinder Server Edition.

With DIAdem DataFinder, you can easily search test data stored within your local index; however, needs and requirements change as you expand a data management solution to large groups or departments. To meet these needs, National Instruments developed the NI DataFinder Server Edition specifically designed for large groups and departments. The NI DataFinder Server edition works out-of-the-box by indexing test files either on a server or a network with no need for IT support or database knowledge. You can also have multiple engineers simultaneously retrieve information from the server to increase productivity among your department.

Figure 4 – My DataFinder is designed for individuals while the NI DataFinder Server Edition is designed for collaboration

Advanced DataFinder Features

Selectively Loading Data After Performing a Search

After using the DataFinder to return filtered data results, you can then selectively load any files based on any of the search parameters that you entered. You can even create a new channel from the properties you just searched for; just select the property header and drag and drop it into the Data Portal. Utilizing the DataFinder to store property information is an easy way to load data points from multiple files to find outliers and correlate numerous test runs.

Figure 5 – Creating a new channel consisting only of maximum values that exceed 52

Finding the Range of a Property with One Click

As the DataFinder indexes the metadata of your files, it keeps track not only of the property name, but also the property value. When using the Advanced search, you can click on the last button of a query to bring up a dialog box that will show the range of the selected property. You can use this setting to set a threshold for your search or to look for the channel that has the absolute maximum value.

Figure 6 – Finding the range of a property with one click

This feature inherits properties. In this case, the first property is to find file names that begin with TR_*.  The second property where we are viewing the value range, Maximum, is only looking in the files that begin with TR_*.

Additional Resources:

The NI DataFinder is one of key features of DIAdem that will help you accelerate your productivity when postprocessing data. To learn more about the DataFinder or DIAdem, you can explore the following links.

NI DataFinder
DIAdem Help: Defining Search Areas
DIAdem Help: Mining External Data
DIAdem Help: Sorting Search Results
DIAdem Help: DataFinder
DIAdem Help: My DataFinder
How to Set Up a DataFinder Federation