You can use a network drive or source code control (SCC) system to transfer test system files to test station computers, but you must also run an installer on the test station computer to correctly install the TestStand Runtime and other NI software. Because installers edit the registry, execute files, start Microsoft Windows services, call DLLs during installation, and other related tasks, copying test system files to the test station computer is not sufficient for the test system to function correctly. Because of the complexity of trying to reproduce all the actions an installer performs, NI strongly recommends that you use the NI installers or an installer you build with the TestStand Deployment Utility to install NI drivers and components and the TestStand Runtime.

Updating Test Systems Using a Network-Based Solution

When you create a new deployable image to replace an existing deployable image, delete the previous deployable image from the network drive or SCC system before you copy or check in the new version to ensure that the network drive or SCC system does not include files you removed from the deployable image.

By default, when you create a patch deployable image, the image includes only modified files. Check the patch deployable image into the SCC system or copy the image to the network drive and overwrite files in the existing full deployable image to ensure the network drive includes updated files.

If you use a network drive or SCC system to update test systems you deploy to test station computers, design the update process to manage network outages and race conditions.

Network Outages

When a test station computer cannot connect to other computers on the network during an update process, the test system might not update correctly with all the required files. For example, to change the prototype of a code module, you must update the code module and the sequences that call the code module. If the test station computer accesses and updates only the code module file but not the sequence files because of a network outage, the existing sequence files on the test station computer cannot load the new code module with the updated prototype, which causes failures in the test system.

You can prevent partial updates by using a temporary directory on the test station computer to store all the updates before moving the updates to the test system directory. You can design the update process so a network outage during the process of updating the temporary directory generates an error and does not copy the partial updates from the temporary directory to the test station directory, which leaves the functional test system unchanged.

Race Conditions

Race conditions occur when the results of a process depend on the order in which tasks complete. To avoid race conditions when you use a network-based solution to update test systems, you must make changes to the files in the central network location before the test station computers access the files.

For example, if you change a sequence file and the related code modules on a network drive just after a test station computer loads the sequence file but before TestStand loads the code modules, TestStand loads the old sequence file and the new code modules at the start of execution. This action might lead to incorrect test results if the new code modules require limits the old sequence files do not contain or might lead to errors executing the sequence file if the code module prototypes do not match the prototypes the sequence file expects. Race conditions can also occur when sequences load code modules dynamically because old sequence files might load incompatible new code modules.

You can avoid race conditions by ensuring that the test system does not load sequences during the update process. For example, you can avoid performing updates to files in the central network location during production testing or you can transfer updates to a temporary directory and then to the test system directory only when the test system unloads all the files.

Validating Network-Based Deployments

For cases where frequent updates are needed, a Continuous Integration and Delivery (CI/CD) approach can help ensure that the latest code is available on test computers, while still ensuring the code has been properly tested and validated. With a CI/CD approach, you automate much of the procedure to build, test, and deploy updates to test code to reduce overhead as much as possible. You can use third party tools, such as Jenkins, to implement this type of automation. For more information on using Jenkins to automate TestStand deployments, refer to the Getting Started with TestStand and Jenkins tutorial.

For more tightly controlled environments, validation requirements are typically very stringent, and therefore test system updates cannot occur frequently. For these test systems, define a separate repository or trunk where developers make changes and add new features to the test. Once the latest version of the test system has been finalized and validated, developers create a versioned branch of the trunk for use on production machines. This branch should be locked within source code control to prevent any changes to validated code. Once the new branch is completed and validated, create a new versioned branch containing the code.