National Instruments designed a benchmarking utility that captures performance profiles for RT targets while they run a lightweight, object-oriented application designed to access large quantities of shared variables at high rates. The following figure highlights the architecture of the benchmarking utility. The main components are a test supervisor application that runs on the Windows host computer and Device Under Test (DUT) application that runs on the RT target.
Test Supervisor Application
A Windows PC is responsible for supervising and coordinating the benchmarking of the DUT. The primary responsibilities of the test supervisor application include:
- Configuring the DUT for testing
- Managing deployment of shared variables
- Issuing commands to the DUT application
- Acting as a shared variable accessor during a benchmark
- Receiving, analyzing and logging data from the DUT application
The DUT application runs on any RT target. The application is a lightweight, object-oriented and utilizes a state machine architecture. The primary responsibilities of the DUT application include:
- Waiting for and executing instructions sent by the test supervisor
- Hosting shared variables
- Programmatically opening, writing and closing shared variables
- Acquiring CPU performance data during a benchmark
- Sending data back to test supervisor
The benchmarking utility generates a performance profile by executing a series of tests that sweep across a range of shared variable quantities and update frequencies. During each test, the DUT writes to shared variables hosted locally while the test supervisor access those shared variables at the same frequency. While the shared variable communication is executing, a parallel thread in the DUT application collects CPU usage information using the NI System Configuration API.
For additional details, please see the Shared Variable Benchmarking Utility homepage.
Summary of Test Conditions
These tests were executed in a controlled environment which used the following conditions across all tests:
- Private Gigabit Network
Benchmark data was executed with the test supervisor and DUT on a gigabit LAN with zero non-test traffic.
- Lightweight DUT Application
To obtain performance profiles that accurately describe how shared variables affect performance, a lightweight DUT application was designed to minimize processor utilization from any functions other than shared variable access.
- Maximum Library Size
Though several thousand variables were deployed in some tests, the maximum library size was limited to 500 variables. It is good programming practice to separate large numbers of shared variable into multiple libraries.
- Double-Precision Floating-Point Data Type
All benchmarking tests were executed using shared variables of the largest non-custom data type native to LabVIEW.
- Single Shared Variable Accessor
Only a single client application accessed shared variables hosted on the target.
- Access via LabVIEW Shared Variable API
Both the DUT application and test supervisor application accessed the shared variables using the LabVIEW Shared Variable API.