Defining Data Integrity Rules

Define data integrity rules to ensure that all data entering the SystemLink file service complies with the metadata requirements for your data analyses.

1. In a text editor, create a JSON file that defines the required metadata at the root, group, and channel level for all TDM data entering the file service.
   

   Tip  

   Update the file version when updating a definition file. DataFinder adds the "FileVersion" property to the checked files to indicate which version of the definition file they comply with. This enables you to search for data that complies with a specific version of the definition file.

   Refer to the code example below for a metadata definition file at the bottom of this page.
2. Upload the JSON file to the DataFinder that indexes the file service.
   1. Select the Data Integrity Rules tab.
   2. Upload the metadata definition from your disk and click OK. The DataFinder instance checks all new files entering the file service for compliance with the metadata definition.
   3. To remove the file, click Remove required metadata definition.
3. Optional: Reset the entire index to execute a compliance check on already indexed files in the file service.
   1. Select the Index tab.
   2. Click the arrow next to Reset Index and choose Entire Index. The DataFinder instance reindexes all files in the file service and checks them for compliance. This process may take a while. During the check, DataFinder adds and optimizes two properties:

      Added Properties	Values
      RequiredMetadata~Compliant (searchable in Data Navigation)	0 = no 1.0 = version of the metadata definition file with which the metadata complies
      RequiredMetadata~ErrorText (not searchable in Data Navigation)	Missing keys: metadata1, metadata2, ...

{
"FileVersion": 1,
"Description": "Required metadata keys for SystemLink (this line is ignored)",
"RequiredMetadataVersion": 1.0,
"RequiredMetadataKeys": {
"root": ["product_identifier", "product_type", "product_design_step"],
"group": ["package", "dut_id"],
"channel": ["ResultStatArithMean", "ResultStatMax", "ResultStatMin"]
}
}