PDF1. PCI Interrupt Lines and Routing
The PCI specification defines four physical interrupt lines per bus: PIRQA, PIRQB, PIRQC, and PIRQD (where PIRQ stands for Peripheral Component Interconnect Interrupt Request). Since the PXI (PCI eXtensions for Instrumentation) platform is based on PCI, this also applies to PXI systems.
All PIRQ lines are routed to each slot in a chassis, but most I/O modules make use of only one. Consider the diagram of the interrupt pins on each I/O module connector in Figure 1.
Figure 1. Each PCI (or PXI) module connector exposes four interrupt pins: INTA - INTD, although most devices make use of only INTA.
Since many modules make use of only the INTA pin, if INTA were always wired to the same PIRQ line, then many devices in the system would share the same interrupt line and interrupt latency would be increased. For this reason, backplane/motherboard designers commonly alternate which PIRQ trace in the backplane is connected to each module slot INT pin (see Figure 2 for an illustration). In practice, this routing varies based on manufacturer and board model; the same is true for National Instruments PXI chassis.
Figure 2. Since most PCI or PXI devices make use of only the bottom (INTA) pin, PCI IRQ (PIRQ) backplane lines are commonly wired in an alternating manner between module slots to spread devices between interrupts.
In PXI systems with the NI Real-Time Hypervisor installed, this routing is important. For performance reasons, I/O modules must be assigned to one OS or the other (most commonly Windows or LabVIEW Real-Time). Since each OS with access to an interrupt line must acknowledge interrupts on that line, if a general-purpose OS such as Windows were to share an interrupt line with LabVIEW Real-Time, then LabVIEW Real-Time may need to wait on that OS to acknowledge incoming interrupts. This has the potential to increase jitter, and is not acceptable for hard real-time systems.
For this reason, in Real-Time Hypervisor systems, each of the four PIRQ lines are "owned" by only one OS. This means that if PIRQA is assigned to Windows, then any slot that routes PIRQA for use with an I/O module (commonly the INTA pin) must also be used with Windows. Effectively there are two major side-effects caused by the internal interrupt routing details on Real-Time Hypervisor systems:
- Because several slots may have INTA tied to a specific PIRQ line, and that PIRQ line must be assigned to only one OS, it is not possible to assign any number of I/O modules to each OS. Instead, there are certain supported configurations (e.g. 3 on LabVIEW Real-Time, 4 on Windows).
- Due to the PIRQ lines being rotated throughout the PXI backplane (see Figure 2), if a specific PIRQ line is assigned to a certain OS, then only certain module slots will connect to that PIRQ line via INTA and enable I/O modules to be used with that OS. Basically, I/O modules may need to be moved around in a chassis to be used with the OS of choice.
The following sections explain some additional factors related to PXI/PXI Express interrupt line routing that can also constrain I/O to OS assignments in Real-Time Hypervisor systems. In most cases it is not necessary to fully understand all of these factors; you can skip to the "Validing I/O to OS Assignments in Real-Time Hypervisor Systems" section for instructions on how to plan for and configure NI Real-Time Hypervisor I/O to OS assignments.
2. Complicating Factors
Reserved Interrupt Lines
In Real-Time Hypervisor systems, certain devices such as the video interface onboard PXI controllers require the use of a PIRQ line, and must be assigned for use with Windows/Linux. Therefore, it is common for PIRQA to be assigned to Windows/Linux and not user-changable on Real-Time Hypervisor systems. This also means that all all I/O module slots that map PIRQA to a used INT pin (commonly INTA) must be used with Windows/Linux.
PCI-PCI Bridging
Some PCI and PXI modules make use of a PCI to PCI bridge to incorporate multiple end devices (such as Ethernet and GPIB) in a single module. Bridges may also be used on modules with single end devices that share common designs with those with bridges; this is the case with many NI PXI Ethernet modules. Finally, bridges are used on MXI interfaces that extend the PCI/PXI bus.
When using a PCI-PCI bridge, the INTA-INTD lines may be routed in various ways to the end devices or expansion chassis; this is implementation-specific. This means that it is more likely for I/O modules with bridges to make use of the INTB-INTD lines, instead of just INTA as is the case with most modules.
PCI/PCI Express Hybrid Systems
In National Instruments chassis that support PXI Express controllers (based on the PCI Express standard), both PXI and PXI Express modules are typically supported; depending on the chassis a combination of PXI, PXIe, and hybrid slots may be present.
PXI-only slots in combo chassis use INTA-INTD pins to connect to PIRQA-PIRQD traces as with traditional PXI chassis. PXI Express slots, on the other hand, do not connect to physical PIRQA-PIRQD traces; instead a serial protocol is used to send data between each module and the controller. PXI Express modules use transmit and receive lines to communicate interrupts to the controller, and these interrupt signals are then mapped to PIRQA-PIRQD signals using BIOS information.
Hybrid slots in PXI/PXIe combo chassis take advantage of different PIRQ signals depending on whether they are used with PXI or PXI Express modules. PXI modules connect to physical PIRQ traces via the INTA-INTD pins in hybrid slots, and PXI Express modules communicate interrupts to controllers via transmit and receive lines, which are then mapped to PIRQA-PIRQD signals.
3. Message Signaled Interrupts (MSIs)
Some National Instruments PCI Express I/O modules, including X Series devices, communicate interrupts without using the PIRQA-PIRQD signals. Instead, they store interrupt information in special memory locations and communicate interrupts by reading to and writing from those memory locations. This technique is used with many PCI and PCI Express based devices and these communications are referred to as Message Signaled Interrupts (MSIs).
When NI I/O modules that use MSIs are used in a Real-Time Hypervisor system, in some cases they can be used independent of the interrupt line constraints mentioned above. You can read the Resource Partitioning topic in the NI Real-Time Hypervisor online help for instructions on how to verify that a device uses MSIs (http://zone.ni.com/reference/en-XX/help/372833B-01/rthyperhelp/hv_resourcepartitioning/). When using an MSI-capable device with the Real-Time Hypervisor, you can also inform the Real-Time Hypervisor Manager of this fact to remove some constraints associated with using the PIRQ lines (enable more I/O to OS assignment combinations).
Note that Windows XP does not support the use of MSIs, but LabVIEW Real-Time and Red Hat Enterprise Linux do.
4. Validating I/O to OS Assignments in Real-Time Hypervisor Systems
Prior to Purchasing a Real-Time Hypervisor System or Additional I/O Modules
If you have not yet purchased a Real-Time Hypervisor system, or have purchased a Real-Time Hypervisor system and are considering purchasing additional I/O modules, it is a good idea to verify that all of your I/O modules can be used with the desired OSs before ordering.
To assist with this task, National Instruments has incorporated logic into the online PXI Advisor (http://ni.com/pxiadvsior) that enables you to choose which OS you would like to use I/O modules in your configuration with, and then validate those assignments to ensure that they are possible prior to ordering. If you are ordering a new Real-Time Hypervisor system, use the PXI Advisor to select the hypervisor controller, chassis, and I/O modules of choice, and then use the Hypervisor I/O to OS Assignment Utility when prompted. If you are considering adding one or more additional I/O modules to an existing Real-Time Hypervisor controller, then choose the hypervisor controller, chassis, and I/O modules that you are currently using -- then add in the new I/O modules and run the Hypervisor I/O to OS Assignment Utility when prompted.
If you are working with custom I/O modules, 3rd party modules, or more complex systems that are not represented in the online PXI Advisor, please contact National Instruments support so that an Applications Engineer can help you validate your desired I/O to OS assignments with your complete system in mind.
After Purchasing a Real-Time Hypervisor System, with I/O Modules
After purchasing a Real-Time Hypervisor system, you can boot into Windows/Linux and change your I/O to OS assignments at any time using the Real-Time Hypervisor Manager. If you have a new I/O module in hand, shut down your system and insert the module, then boot into Windows and run the Real-Time Hypervisor Manager. Choose the OS to assign each I/O module (including your new module) to, and follow instructions for relocating modules in the chassis if needed.
The Real-Time Hypervisor Manager will display a broken Apply arrow and a warning if your desired I/O to OS assignments can not be achieved given interrupt routing constraints.
5. Related Resources
>> National Instruments PXI Advisor (order a complete hypervisor system or validate I/O to OS assignments)
>> NI Real-Time Hypervisor Architecture and Performance Details
>> NI Real-Time Hypervisor Help
>> National Instruments Technical Support
