You can often improve test speed by utilizing parallel testing to complete multiple tests simultaneously. TestStand provides features to help you parallelize the testing of a single UUT, or for testing multiple UUTs simultaneously.
Parallelizing a Test Sequence
When testing a single UUT, you may be able to test multiple portions of the system at the same time. For example, consider the test sequence for a mobile device. Tests for each component such as the cellular data, user input, and audio systems can each be implemented in separate subsequences. Rather than calling each sequence sequentially, you can configure the sequence call step to call them asynchronously to speed up the test.
To specify that a sequence call should be executed asynchronously:
- Select a sequence call step, and click the Module tab.
- In the Execution Options menu select the Use New Thread or Use New Execution option.
Calling a sequence in a new thread allows you to test different parts of a UUT simultaneously
When configuring asynchronous sequence calls, consider the differences between using a new thread versus a new execution:
|Shares the same results collection and report as the caller
||Has its own results collection and report
|Is executed directly
||Can be executed using a process model entry point
|Shares sequence file global values with caller
||Has a new copy of Sequence File Global values
|Is terminated or Suspended with the caller
||Is independently Terminated or suspended
Typically, you should use the new thread option for related tests within a test sequence. Using a new execution is more appropriate for more independent functionality, such as a status monitor which should run independently of the test sequence.
For more information on choosing whether to use a new thread or execution, refer to When to Run a Sequence in a New Execution versus in a New Thread.
Obtaining Results of Asynchronous Sequence Calls
To obtain the results of your asynchronous subsequences for reporting or database logging, use Wait steps at the end of the launching sequence to wait for the asynchronous sequence calls to complete. Once the subsequence thread completes, TestStand attaches the results of the asynchronous subsequence to the Wait step results, making them available for report generation and database logging. To wait on a sequence execution or thread, select Execution or Thread in the Wait for control and specify the execution or thread to wait for. Keep in mind that adding this wait may cause a delay in execution if the calling sequence completes before the thread, so only use this approach when you need the results of the new thread
Use a wait step to obtain the results of an asynchronous sequence call
Testing Multiple UUTs in Parallel
In addition to using asynchronous calls in your test sequence, TestStand allows you to test multiple UUTs in parallel using the Parallel and Batch Process Models. These process models create multiple executions which each run the test sequence on a separate UUT. You can change the process model for the current station or for an individual test sequence file.
Test multiple UUTs simultaneously using the parallel and batch process models
For a demonstration of how parallel testing can improve test times, refer to the Parallel Test Strategies Demo included with TestStand
Choosing Between the Parallel or Batch Process Model
While the Parallel process model allows you to start and finish tests of UUTs at different times, the Batch process model is designed to start and finish the test sequence on all UUTs at the same time.
- The Parallel process model is ideal for testing multiple UUTs using separate test fixtures, where each UUT is tested independently.
- The Batch process model ideal for testing UUTs that are tested as a group using a common test fixture or test chamber, where testing must be synchronized for all UUTs
In addition to starting and ending each batch test at the same time, the batch process model allows you to use batch synchronization steps and settings to further synchronize testing across UUTs in the batch. If certain portions of the test should be run at the same time for all UUTs, you can define synchronized sections for a single step using the step settings, or for multiple steps using the batch synchronization step type.
Use batch synchronization to ensure that certain portions of the test are synchronized for all UUTs in the batch
For all types of batch synchronization sections, all sockets enter and exit the section together. You can further configure the synchronization to execute sequentially or only in a single thread.
For more information on batch synchronization, refer to the Synchronization Step Types – Batch Synchronization example
Hardware Considerations for Parallel Testing
When testing multiple UUTs in parallel, the available testing hardware can become a performance bottleneck. When using a shared hardware resource, you must ensure that only one thread is accessing a shared hardware resource at any given time to avoid resource conflicts. Typically, you use the Lock settings or step types to reserve a shared resource. However, if multiple threads are waiting on a single resource to complete a test, many of the potential improvements of parallel test will not be realized. To mitigate this, you can use the following approaches:
- Use the Auto Schedule Step types to allow other tests to execute while waiting for resources.
- Use the Execution Profiler to find hardware that is causing the bottleneck and add additional test hardware.
Using Auto Scheduling
The Auto Schedule step type allows you to configure a set of tests that can be executed in any order to optimize test time and hardware utilization. When an auto Scheduled section is executed using the parallel or batch process model, each socket executes the first section which does not require a reserved resource. For this reason, the execution order can vary between different executions of the same test
Use the auto scheduler to optimize hardware utilization when execution order is not important
Use this approach when the execution order of tests is not important. If your test requires the test results to be presented in a specific order, do not use auto scheduling.
Using the Execution Profiler
The execution profiler tool provided with TestStand is useful for identifying which test hardware is limiting the execution speed of the test system. Launch the profiler from the Sequence Editor using Tools » Profile Execution.
The profiler allows you to see how much time each hardware resource is active, allowing you to make informed decisions on the impact of adding hardware to the test system. In the example profile below, the DMM is fully utilized while the Scope is only 66% Utilized. Based on this profile, adding a second DMM or a DMM with more channels would improve the test time for this sequence.
The execution profiler allows you to visualize which resources are potential bottlenecks in your test configuration