Reduce Development Time With the NI TestStand Sequence Analyzer

Publish Date: Aug 11, 2011 | 5 Ratings | 3.00 out of 5 | Print

Overview

Static code analysis is the process of improving the quality of code without actually running the code (in which case it is called dynamic code analysis). Analysis is important to ensure the quality of code, where code quality encompasses not only performance, but readability, reusability, and maintainability. Prior to the release of NI TestStand 2010, developers had no way to apply software engineering best practices and ensure the quality of their code without manual code reviews, stringent source code control (SCC), and simple trial and error.

The NI TestStand Sequence Analyzer entirely changes the approach to NI TestStand sequence development and drastically reduces the amount of time developers will spend identifying erroneous code or enforcing development best practices.

Table of Contents

  1. NI TestStand Sequence Analyzer Overview
  2. Configuring an NI TestStand Sequence Analyzer Project
  3. Using Built-in Sequence Analyzer Rules
  4. Creating Custom Rules and Analysis Modules
  5. Ensuring Flexibility Through User-Configurable Rules
  6. Sharing or Deploying Rules Among Development Teams
  7. Analyzing Sequences and Reporting Results
  8. Analyzing LabVIEW Code Modules
  9. Learn More Today

1. NI TestStand Sequence Analyzer Overview

The NI TestStand Sequence Analyzer operates similarly to familiar static code analyzers. Using the Sequence Analyzer, you can quickly gain edit-time insight into your sequences and files. Edit-time insight helps you minimize errors, refactor code for performance, and ensure maintainability over time. 

To identify the most common situations that cause run-time failures, the Sequence Analyzer uses a built-in set of rules and analysis modules to examine specified files to quickly isolate errors, enforce conformity to development guidelines or best practices, and deliver statistical information to the developer.

Never before seen in the automated test industry, the Sequence Analyzer provides both a utility for statically analyzing sequences and a framework for developing and tailoring the Sequence Analyzer to specific teams or application requirements. Violation of rules can result in one of three severity levels, as shown in Figure 1, allowing categorization of results. After completing your analysis, you can export analyzer results to an XML report file for viewing in an external application. 

Figure 1. The NI TestStand Sequence Analyzer reports results from rule violations in three severity levels – Error, Information, and Warning.

Back to Top

2. Configuring an NI TestStand Sequence Analyzer Project

You can configure the analysis of sequence files or directories using an NI TestStand Sequence Analyzer project, a file configured and stored on disk using the extension *.tsaproj. Sequence Analyzer project files store settings for analysis – analyzer rules and whether they have been enabled or disabled, severity levels, analysis options, and the location of files or directories to analyze.

Figure 2. Sequence Analyzer Projects store information on rule configuration, analysis behavior, and which sequences to analyze.

Back to Top

3. Using Built-in Sequence Analyzer Rules

NI TestStand installs out of the box with more than 20 default rules that do everything from provide information about the context of an execution (for example, the total number of steps included in the execution) to check for the most common mistakes that lead to run-time errors (for example, code modules that cannot be found on disk).

For detailed descriptions on all the built-in Sequence Analyzer rules please see the TestStand Help topic - TestStand Sequence Analyzer Rules Descriptions.

Back to Top

4. Creating Custom Rules and Analysis Modules

The built-in Sequence Analyzer rules offer a good foundation for expediting development time, but the NI TestStand Sequence Analyzer also provides a framework for creating and configuring custom rules and corresponding analysis modules. Using this framework, you can expand the Sequence Analyzer’s capabilities to check for more obscure conditions that lead to errors as well as ensure that teams collaborating in development always conform to approved style guidelines and coding best practices.

To add custom functionality to the NI TestStand Sequence Analyzer, creation and configuration of two components is required: rules and analysis modules. Within the Sequence Analyzer Options dialog box, you can access all of the rules configured on the system as well as add or remove any custom rules.

Figure 3. Use the Sequence Analyzer Options dialog box to access and configure all rules and analysis modules in the system.

A rule defines a single task to perform during analysis, such as ensuring that the prototype of each code module is up-to-date or that each expression specified evaluates correctly and does not include any syntax errors. Analysis modules specify the types of objects to analyze for the rule, specify the functions in DLLs or VIs that the Sequence Analyzer invokes to implement rules, and generate the message that the Sequence Analyzer stores in the project.

Figure 4. In the Edit Rule dialog box, you can configure the severity of the rule, provide descriptive information, and specify the rule ID to identify the rule on the system.

When creating custom analysis modules, you can copy the provided example templates located at <TestStand>\Components\Analyzer\Templates to the <TestStand Public> directory to ensure that the prototype of the module matches Sequence Analyzer requirements. By starting with a template, the only custom work you must do to complete the module is use the Sequence Analyzer API to perform rule checking.

After creating a custom analysis module and configuring a custom rule, you can associate your analysis modules with the Sequence Analyzer using the Edit Analysis Module dialog box, as shown in Figure 5, to configure the path to the module as well as the NI TestStand objects, types, or transitions on which the module operates.

Figure 5. Use the Edit Analysis Module dialog box to configure objects, types, and transitions for the module.

Back to Top

5. Ensuring Flexibility Through User-Configurable Rules

When creating custom rules and analysis modules, you may encounter situations where the specific parameters defining the rule are variable. For example, in the case where a rule is meant to enforce that a specific prefix is used to differentiate custom step types from default NI types, the exact prefix requirement may change over time or between projects.

You can hard code variable values into analysis modules; however, this is not a flexible architecture if the values are expected to potentially change (or even be user-configurable) over time. If the specific parameter changes over time, the analysis module needs to be updated and then potentially revalidated.

Instead, you can use the Sequence Analyzer framework to associate rule settings or configuration modules that automatically populate rule parameters. As shown in Figure 6, rule settings are simple, built-in configuration tables that specify numeric, text, or Boolean parameters. If you need a more customized look and feel for your configuration settings or need to manipulate more complex data types, you can use configuration modules – actual code modules on disk that manipulate the Analyzer API. Think of these modules like edit substeps for custom step type creation.

Figure 6. Configure the rules with rule settings that automatically update specific rule parameters without requiring changes to analysis modules.

Back to Top

6. Sharing or Deploying Rules Among Development Teams

As shown in Figure 7, you can import and export custom rules and your configurations to share them between development teams or deploy them to target machines. This makes standardizing on rules during team-based development drastically simpler; you can export and store rules in a network location for all developers to access. After exporting custom rules, all analysis modules (DLLs or VIs) and other supporting files must also be copied and shared.

Figure 7. You can import or export custom rules to standardize on the same set of rules when developing as a team or so that rules can be copied to deployment machines.

You can also use the Sequence Analyzer Install Utility to automatically install or uninstall custom rules and analysis modules without requiring user interaction. This utility, located in the <TestStand>\Bin directory, is called AnalyzerInstallUtil.exe.

Back to Top

7. Analyzing Sequences and Reporting Results

After you have configured NI TestStand Sequence Analyzer Projects – including all enabled rules and files to analyze – you can interactively begin sequence analysis in one of two ways. First, you can perform sequence analysis on demand. This option is ideal for one-time or occasional sequence analysis performed only when a developer specifies. If you wish to more consistently ensure that sequences pass analysis before execution, toggle the Analyze File Before Executing option in the toolbar, as shown in Figure 8.

Figure 8. When you toggle the Analyze File Before Executing option, the Sequence Analyzer executes the current project before every execution.

Once analysis is complete, you see a list of results that can be grouped and organized in several ways for understanding. You can ignore or mark as complete analyzer result messages and restart analysis immediately if necessary.

If you want a more portable version of analysis results, you can export messages in the form of an analysis report, which is an *.xml report that is specially formatted to enable review of messages, organization of reported messages based on severity, and examination of system rules. You can share these analysis reports with colleagues, save them to prove conformity to development practices, and view them in an external application. You can even customize the style sheet to format these reports to suit more specific application requirements.

Figure 9. You can distribute analysis reports among colleagues and view them in an external application.

Back to Top

8. Analyzing LabVIEW Code Modules

When analyzing code modules based on the NI LabVIEW graphical development environment, consider the LabVIEW VI Analyzer Toolkit and LabVIEW Desktop Execution Trace Toolkit for static and dynamic code analysis, respectively. Using these tool kits, you can drastically improve the quality of your LabVIEW code modules. From automating application debugging and code review to profiling the execution of VIs and identifying problems that could negatively impact execution speed, you can optimize the performance of your NI TestStand systems for maximum throughput. To learn more about these and other National Instruments tools for software engineering and code quality assurance, visit ni.com/largeapps.

Back to Top

9. Learn More Today

Download a free evaluation of NI TestStand.

See what else is new in NI TestStand.

Learn more about software engineering and large application development.

Back to Top

Bookmark & Share


Ratings

Rate this document

Answered Your Question?
Yes No

Submit