Automated test systems are usually designed to maximize throughput by minimizing the time required to verify a unit under test (UUT). Test system architects spend a considerable amount of time optimizing test system performance. In a typical test system, the amount of time required for report generation depends on a number of factors, including available resources (processor and memory), the format of the report to generate, the number of results, and additional data to log.

Recommended Report Options

NI benchmarks demonstrate the following patterns based on the report format, the number of step results or the size of the data sets you log, the number of cores in the test computer, and other options you configure in the Report Options dialog box:

  • When you log 500 or fewer step results on a single- or multi-core computer, ASCII reports have the least negative impact to throughput, and XML reports have the most negative impact to throughput.
  • When you log more than 500 step results or large data sets on a single- or multi-core computer, TSR reports have the least negative impact to throughput, and XML reports have the most negative impact to throughput.
  • Asynchronous report generation on a single-core computer does not improve throughput, except in cases where a substantial amount of sequence execution time is spent waiting on operations that do not use the CPU, such as instrument I/O.
  • Asynchronous report generation on a multi-core computer improves throughput for all report formats.

The following table summarizes the recommended options you configure in the Report Options dialog box, depending on your system configuration, when you optimize for test system throughput:

Report Option Single-Core Computers Multi-Core Computers
Value for ≤ 500 Step Results Value for ≥ 500 Step Results Value for ≤ 500 Step Results Value for ≥ 500 Step Results
Format ASCII, TSR*, ATML, HTML, XML (best to worst) TSR, ASCII, ATML, HTML, XML (best to worst) ASCII, ATML, TSR, HTML, XML (best to worst) TSR, ASCII, ATML, HTML, XML (best to worst)
Asynchronous False False True True
On-the-Fly Report False False False False
Only Display Latest Results False False False False
Note *The TSR format generates a compact raw results file and, if used, requires the TestStand Offline Results Processing Utility to generate the final report.

Tradeoffs

Consider the impact to the following requirements when you optimize for test system throughput:

  • Minimize report file size
  • Include all necessary report content
  • Interoperate with other processes and systems
  • Support post-failure information recovery
  • Generate reports immediately for the current UUT before testing the next UUT
  • Generate multiple reports for each UUT
  • Generate and view the report during test sequence execution