What Is Hardware-in-the-Loop (HIL)?

The significant impact of time to market became highly visible in the 1980s through groundbreaking work by former McKinsey consultant Donald G. Reinertsen, indicating that “six months delay can be worth 33 percent of lifecycle profits”i caused by product development. Furthermore, the launch and destruction of the Ariane 5 rocket on June 4, 1996—known as one of the most infamous software bugs in history—taught the engineering world the importance of embedded software quality and the need for proper testing. The result was a loss of more than $370 million.ii These numbers are clear evidence of the consequences of delays and need for shorter time to market as well as cost reduction by identifying bugs and errors through proper testing earlier in the product design and development lifecycle.

It is key to use technology that catches defects before the production begins. It’s increasingly crucial to make test an integral and strategic asset—as part of the full product design and development cycle and not just an afterthought or even a showstopper at the very end of the process. This understanding has led to the development and implementation of methods and strategies known as front loading test, shifting left, model-based design, and anything-in-the-loop (XIL) testing. This white paper has a strong emphasis on HIL testing as one of the crucial testing methodologies in embedded software validation.

Before diving straight into HIL testing, it is paramount to look at the actual product to be developed: the device or system under test (DUT/SUT). The products of today and tomorrow have clearly become far more software-centric and complex. When considering examples like Wi-Fi routers, dishwashers, food processors, and so on, they are all based on embedded software or firmware now. Cars, airplanes, and smartphones are prime examples—not only for being highly software defined, but also highly complex—because they are a system of software-based systems. Ultimately, they all do have in common that software is the defining element of their current feature set and future improvements to be released. Updatable apps on a smartphone that turn it into a multitool of thousands of features and functions are the clear embodiment of this trend, which has evolved into the undeniable expectation of today’s customers for almost every other product as well.

Apps, firmware, and software in general cannot stand alone, as they need hardware to be executed on. Typically, the hardware is a computer or processor—or, simply speaking, a controller. This means a product or system (including systems of systems) requires the combination and integration of hardware (controller) and software, which then becomes the DUT. Moreover, the controller also requires its environment to interact with. For example, the controller of a washing machine needs to check if enough water is present, measure the temperature of the water, control the speed (RPM) of the washer drum, and so on. Therefore, the controller requires the connection to sensors (temperature, RPM) and actuators (heater, electric motor) providing data or feedback as inputs and sending commands or setpoints as outputs. These are just the very basic components of control loop design theory, as depicted in Figure 1. Here, the environment around the controller is typically referred to as the plant.

Figure 1. Fundamentals of Control Loop Design

When looking at the product design and development cycle, it is apparent that not all components of the control loop will be available at the beginning of a project. Even more so, it’s also unrealistic to have all the necessary hardware and software components ready and available at the early stages of development. This is where simulation and model-based design start to fill in the gaps.

Figure 2 shows the product development lifecycle in a typical V-Diagram representation, with its different stages from design to prototyping, to software and controller testing, to physical testing. It also shows the testing methodologies of model in the loop (MIL), software in the loop (SIL), and hardware in the loop (HIL).

Figure 2. The V-Diagram: From Anything in the Loop (XIL) to Physical Test

Digital simulation and model-based design are key across the entire design process because they:

• Allow for development and testing before required physical components are available
• Increase test coverage, create faster design iterations
• Improve speed by minimizing the number of redundant (physical) tests
• Accelerate product quality testing for corner cases and all possible scenarios

Simulation and model-based design enable the start of testing much earlier (referred to as front-loading test) through test methodologies like MIL, SIL, and HIL. As shown at the bottom of Figure 2, components of the control loop are being replaced step-by-step when moving from right to the left on the V-diagram (also referred to as shift left), while moving to the right in line with the actual product design and development lifecycle, components get replaced with the actual real hardware and software as they become available.

In addition, virtual testing maximizes test coverage and reduces the amount of expensive and time-consuming physical tests. A rough estimate for the number of defects found in embedded software is 10 to 20 for every 1,000 lines of code. This may not seem like a lot at first glance, but when looking at how many lines of code are now in everyday systems, the estimated number of defects can be incredibly high. Considering medical devices, like pacemakers, which have 80,000 lines of code (~800 to 1,600 defects), or an MRI scanner with 7,000,000 (~70,000 to 140,000 defects), the impact of the expansiveness in software complexity becomes apparent—and far more serious.

The complexity of these systems is overwhelming to the point that it becomes unachievable to exhaustively test them in the traditional way. Physically testing all scenarios is impossible, and it takes an unreasonable amount of time and money. Moreover, it’s important to be confident that test cases cover all possible real-world conditions. Big failures, disasters, and device recalls are very expensive, as noted in the Ariane 5 example in the introduction of this white paper. But even more so, the negative impact of these consequences on brand imagery and company reputation is simply priceless. Mitigating this risk can be achieved through modeling and simulation. Virtually, it is feasible to test products in all possible, repeatable scenarios and with confidence, so the final physical test is only the last check and not an expensive catastrophic event.

Embedded software testing serves as a methodology for problem detection, connecting design and test teams with a compatible workflow that heavily uses simulation and model-based design.

Figure 3. MIL, SIL, HIL, and Physical Test in Control Loop Representation

The first stage, MIL (quadrant 1 in Figure 3), simulates everything, including the controller and the entire plant (environment) around the controller.

During the second stage, SIL (quadrant 2 in Figure 3), software engineers generate target-ready code only from the control model, replacing the block and creating a software prototype while the plant is still simulated. These first two stages enable test execution using simulation and models only barely require real, physical hardware components.

The third stage, HIL (quadrant 3 in Figure 3), is pivotal in this methodology. The code is deployed and executed on a physical, hardware-based control unit (a rapid control prototyping platform or a produced controller), allowing testing of all possible real-world scenarios by using the simulated plant and again before moving to (final) physical testing (quadrant 4 in Figure 3).

Therefore, HIL can be seen as the equivalent to the marriage between the chassis and drivetrain of a vehicle with its body on a vehicle production line. It is essential to execute the developed software code on the actual controller hardware, to make sure that all timing, synchronization, and execution flaws can be identified, which typically are not present when running the embedded code in a SIL environment, for example.

There are great expectations and considerable ongoing developments to further increase the emphasis and utilization of MIL and SIL. But HIL will and must always be the proof point for executing embedded software on the real controller platform—to release safe and secure software-centric products with confidence to the market. In summary, HIL testing provides:

• Safe testing—Avoids risks of damaging expensive or dangerous equipment
• Cost-effectiveness—Reduces the need for full prototypes
• Fast development—Enables early testing before the full system is built
• Repeatable scenarios—Simulate common and/or edge cases and faults reliably

Figure 4. The fish analogy: An HIL system is the equivalent of an aquarium.

HIL is a testing methodology that provides a virtual environment or virtual reality (digital twin) around the physical DUT (embedded software running on a controller) to make it “believe” it actually is connected to the real environment (physical twin). A very simple analogy to this is a fish in an aquarium or fish tank, as depicted in Figure 4. The aquarium perfectly mimics the real-world environment of the ocean, by providing a controlled simulated environment, to make sure that the fish have everything they need to survive (food, water, temperature, salinity, and so forth).

When looking at the technical side and focusing on software-centric products like a DUT, HIL is an embedded software test technique during which real signals from a controller are connected to a test system (plant). HIL simulates reality by using software models and simulation. So, the HIL test system mimicking the real-world environment is the equivalent of the aquarium—and the DUT (physical controller executing embedded software) is the equivalent of the fish. This makes the controller (or DUT) believe it is installed in the assembled product like a washing machine, airplane, or car.

Test and design iterations take place as if they were connected to the real-world system, but through a simulated plant. By using repeatable, virtualized scenarios, engineers can exercise the controller across thousands of conditions without the need to have all real components available, cost, complexity, or scheduling constraints of hardware-based physical testing. The real controller or DUT gets its inputs from the HIL system and sends its outputs back to the HIL system. The HIL system consists of inputs and outputs including bus interfaces, as well as a real-time compute core that runs the HIL application software, including the required plant model(s). To make sure a HIL test system setup fulfills all requirements to provide an extensive virtual reality around the DUT, it needs to consist of at least these core elements, as shown in Figure 5:

• DUT (embedded software running on a controller)
• I/O and buses interfaces to connect to the DUT
• Real-time compute to read, write and control the I/O and communication buses deterministically, as well as execute application software
• Application software to configure the test system, integrate and deploy simulation models, execute a test or multiple test cases (manually and automated) as well as analyze and debug results and behaviors
• Simulation model(s)

Figure 5. Control Loop in an HIL Test System Setup

As previously mentioned, these are just the core elements of a HIL system. Additional elements may also include:

• Fault insertion
• FPGA-based I/O
• I/O switching
• Signal conditioning
• Connections to loads or emulated loads
• Restbus simulation (required inputs from other controllers; for example, in a system of systems)
• Test automation all the way up to continuous integration and continuous deployment (CI/CD) pipelines

The NI platform-based approach to HIL testing provides developers and users with a toolbox of open and flexible building blocks, which allow for heavy customization using commercial of the shelf (COTS) products, and to scale from small benchtop to component and system as well as full integration HIL setups. Users get to decide what to use and how to extend and expand for current and future challenges. They remain in full control of their workflow as well as the development and maintenance of their test assets, while minimizing vendor dependency.

The NI HIL test system architecture offers a comprehensive hardware and software platform addressing the different elements of a HIL system. Its open, customizable toolchain, maximizes the ability to adapt, scale, and expand a HIL test system with a mix of NI, NI Partner, and third-party hardware and software.

PXI is a leading industry-standard hardware platform built on true COTS technology. It delivers best-in-class timing and synchronization capabilities and serves as a foundational element of HIL test systems. PXI also supports a broad range of I/O and bus interfaces—from DC to RF—including FPGA-based technology, general-purpose interfaces, and industry-specific communication protocols.

These I/O and bus offerings can be further extended through the addition of the NI Switch Load and Signal Conditioning (SLSC) platform. SLSC adds custom front-end modules to the signal path between the actual I/O and bus interfaces on the test system side and the DUT. Through standardized, COTS connectivity and form factors, it eliminates the fault-prone process of point-to-point wiring. This simplifies the overall system and allows for signal switching and conditioning, load integration, fault insertion, and more.

NI VeriStand is the core application software for NI HIL systems and embedded software test under real-time conditions. It accelerates the product development lifecycle through configuration-based HIL system setup and control (UI and API), debugging, model integration, real-time stimulus generation, in-product automation as well as extensive custom automation API functionality through NI LabVIEW, C/C++, Python, and more.

Learn more about the NI HIL Test System Architecture here.

Today’s products have become software-centric, and the underlying foundation to the customer expectation is that a product will evolve and improve in the future—providing continued and even new value. Virtual validation test utilizing simulation, model-based design and HIL testing are key methodologies to increase test coverage and product quality. These methodologies also shorten time to market by starting and integrating test into early design and development cycles, while also minimizing the need for costly and time-consuming physical test.

This turns the word “test,” which is often defined and used as the final steps needed to move a design to production and ultimately to market, into a strategic advantage for overall product innovation. HIL elevates testing to more than a checkbox on a project plan. It becomes an integral part of the innovation that makes a design and a company successful (see “six months delay can be worth 33 percent of lifecycle profits”).

Forward-thinking companies are using HIL outside of the traditional road to market testing. Though the long-term goal of HIL is to prevent costly mistakes (as in the Ariane 5 example) in an expensive and/or mass-produced final product, it’s also a design tool that software engineers can use to iteratively test and tweak their software designs. This leads to a vetted product before formal testing even begins. Additionally, software engineers can conceive of and test new ideas quickly, which helps them maximize innovation through timely feedback.

• Explore HIL Test System Architectures
• See How Whirlpool Uses HIL Testing
• Learn How HIL Accelerated Archer Aviation
• See How ZF Validates ADAS With HIL