PDF1. Trade-Off 1 – Averaging versus Repeatability
In either automated design validation or production test applications, averaging multiple measurement results is a common technique to improve measurement repeatability. However, configuring a greater number of averages to improve repeatability comes at the cost of measurement time. In general, total measurement time scales linearly with the number of averages used. Thus, if a single measurement takes 20 ms, configuring 10 averages for the same measurement increases the total time to nearly 200 ms.
At a high level, averaging improves repeatability because it allows nonrepeatable impairments, such as additive white Gaussian noise (AWGN), to effectively cancel from one measurement to the next. To illustrate the effect of averaging on repeatability, you can perform a loopback test using the NI PXIe-5673 RF vector signal generator and NI PXIe-5663 RF vector signal analyzer. For this experiment, generate 802.11g orthogonal frequency division multiplexing (OFDM) signals at 2.412 GHz and an RF power level of -10 dBm. Also, use four different signal types – BPSK (6 Mbps), QPSK (18 Mbps), 16-QAM (24 Mbps), and 64-QAM (54 Mbps) to observe the effect of burst size and modulation scheme on measurement time. Using a payload of 1024 bits, each signal type has a different number of OFDM symbols. For example, the BPSK burst uses 343 symbols while the 64-QAM signal uses 39 symbols. As a result, the burst duration for each signal type also varies. Table 1 illustrates these differences.
Table 1. Modulation Scheme, Burst Duration, and Number of Symbols for Various 802.11a/g Data Rates
The error vector magnitude (EVM) measurement provides the most comprehensive view of a signal’s modulation quality. For each EVM measurement, two built-in methods of averaging are used when reporting the result. For IEEE 802.11a/g bursts, the measurement result is reported as a root mean squared (RMS) of the EVM over each OFDM subcarrier and over every symbol. From Table 1, you should intuitively recognize that given the number of symbols in a burst, the EVM of lower data rates such as 6 Mbps (BPSK) should produce more repeatable measurements than the 54 Mbps burst, given that there are more symbols in longer bursts. This assumption is true only when the EVM is reported as the RMS over the entire burst and not over a specific portion of it. Trade-off 2 investigates the repeatability when analyzing only a partial burst.
In general, you can assume that measurements performed over longer bursts produce more repeatable EVM results. In Figure 1, observe the relationship between the number of averages and measurement standard deviation. These measurements were performed using the NI PXIe-5673 RF vector signal generator and the NI PXIe-5663 RF vector signal analyzer. An RF average power of -10 dBm was used, and the center frequency of both instruments was configured to 2.412 GHz.
Figure 1. Averaging reduces the standard deviation of measurement averages.
Figure 1 shows that the standard deviation of 1000 EVM measurements decreases as the number of averages used in each measurement increases. Note that both the EVM and standard deviation reported in Figure 1 are significantly better than those you might observe on an actual 802.11g transmitter because the source used in Figure 1 is an RF vector signal generator – a product that is fundamentally designed to generate repeatable signals. Thus, consider the results shown in Figure 1 as the repeatability floor. Note that measurement repeatability is valuable only in light of the absolute measurement value. In general, the better the EVM floor of the test instrument, the less repeatability matters. Thus, Table 2 illustrates the EVM results when the measurement is configured for 10 averages.
Table 2. Mean EVM is relatively consistent with modulation scheme.
Table 2 shows that the measured EVM is consistent across all modulation schemes. However, it also shows you can achieve much better standard deviations over longer bursts, where more symbols are measured. For example, while 10 averages achieves a standard deviation of 0.081 dB on a 64-QAM signal, you can achieve the same standard deviation using only five averages when measuring the full burst of a BPSK signal.
In general, you can achieve the lower result standard deviations through averaging only at the cost of a longer measurement time. Table 3 illustrates this relationship for a 54 Mbps burst. Note that measurement time reported includes both a gated power and EVM measurement.
Table 3. Measurement time increases with the number of averages
In Table 3, composite EVM and gated power measurements were performed using an NI PXIe-5663 RF vector signal analyzer and an NI PXIe-8106 controller. EVM is calculated as an RMS over the entire burst, and the reported value is the mean and standard deviation reported over 1000 measurements. Table 3 demonstrates that the relationship between measurement time and the number of averages is almost linear. The NI WLAN Analysis Toolkit uses a technique called asynchronous fetching, which is when the analyzer fetches a new record while the previous record is processed. As a result, you can configure a measurement for multiple averages without incurring a linear time penalty. Notice in Table 3 that with one average configured, an EVM and power measurement takes 9.4 ms. However, with 10 averages configured, the measurement takes only 63.6 ms, which yields 6.3 ms per average.
2. Trade-Off 2 – EVM Over Full versus Partial Burst
A second trade-off that can produce faster EVM measurements in some situations occurs when you configure the instrument to perform the measurement over a partial burst rather than the entire burst. By default, the NI WLAN Analysis Toolkit performs the OFDM EVM measurement as an RMS of all subcarriers over each symbol in the entire burst. Similarly, the NI WLAN Analysis Toolkit reports 802.11b DSSS EVM measurements as an RMS of all chips in the burst. However, there are many instances when measuring only the first portion of the burst can produce repeatable measurement results and save measurement time. In these cases, you can programmatically configure the number of symbols or chips that are used to calculate the EVM measurement.
To illustrate the effect of analyzing a portion burst, consider two different bursts, one using BPSK (6 Mbps) and one using 64-QAM (54 Mbps). As Table 1 shows, the BPSK burst has a duration of 1434 µs with 343 symbols, and the 64-QAM burst has a duration of 176 µs with 39 OFDM symbols. Again, the experiment shows the result of computing EVM measurement time as the mean of 1000 measurements. Each measurement is performed with one average and traces turned off. Figure 2 illustrates the relationship between the numbers of symbols used to compute the measurement and the measurement time for a BPSK burst.
Figure 2. Relationship between Standard Deviation and Symbols Measured for BPSK Burst
Figure 2 shows that you can significantly reduce the measurement time of longer bursts such as BPSK simply by analyzing a portion of the burst instead of every symbol. By using a smaller number of symbols, you can cut measurement time from 40 to 22 ms for this burst. In addition, the repeatability is only slight worse for the faster measurement case.
Obviously, the benefit of measuring only a portion of the burst is that you can reduce measurement time for longer-duration bursts. The reason for this is that the overhead for performing a measurement (memory allocation, driver calls, and acquisition time) makes up a comparatively smaller portion of the overall measurement time for longer bursts. By contrast, shorter bursts (like those for 64-QAM and 16-QAM) are less flexible with respect to the number of symbols used. For example, a 64-QAM burst contains only 39 symbols to begin with. Because you need more than 16 symbols to produce a repeatable EVM measurement, you cannot significantly reduce overall measurement time for 64-QAM bursts. Figure 3 illustrates the relationship between measurement time and the number of symbols used for a 54 Mb/s burst.
Figure 3. Analyzing a Partial Burst Faster for Longer Bursts
All results shown in figures 2 and 3 were achieved using an NI PXIe-8106 controller to produce the fastest measurement times. Note that these results suggest that in some cases, analyzing only a portion of a burst can save valuable measurement time for longer BPSK and QPSK 802.11a/g signals.
Using the WLAN Analysis Toolkit, you can similarly configure IEEE 802.11b EVM measurements to be calculated over a portion of the entire burst as well. Because 802.11b uses direct-sequence spread spectrum (DSSS), EVM is calculated over multiple chips. While the default EVM measurement is calculated over an entire burst, you can configure the WLAN Analysis Toolkit to perform an EVM measurement over as few as 1000 chips.
Figure 4. 802.11b Measurement Time is by Configure EVM over Fewer DSSS Chips
From figure 4 observe that reducing the number of chips used to perform the measurement can reduce measurement time from 300 to 170 ms for a 1 Mbps signal burst.
3. Trade-Off 3 – Composite versus Single Measurements
A third strategy you can implement to reduce WLAN measurement time is to perform composite measurements instead of configuring each measurement separately. With the WLAN Analysis Toolkit, you can obtain all time-domain measurements (power versus time, EVM, and frequency offset) with a single composite measurement. Because composite measurements calculate multiple measurement results on a single burst, they are more efficient than when each measurement is performed sequentially.
When using a composite measurement to measure power, you need to consider two methods. With the WLAN Analysis Toolkit, you can measure RF power either over the full burst or as a gated measurement over a smaller portion of the burst. Typical measurement times for each of these individual measurements are listed in Table 4. All the results shown in this table are the mean of 100 measurements configured with one average. In this case, each 802.11a/g EVM measurement is calculated using 16 OFDM symbols. The gated power measurement is performed on the portion of the burst from 20 to 120 µs.
Table 4. Measurement Time of Composite versus Single Measurements for 802.11a/g
Table 4 shows that when you perform key 802.11a/g measurements such as EVM and power as a composite measurement on a single burst, the total measurement time is significantly less than when you perform each measurement singularly. The composite measurement shown in Table 4 is for EVM, gated power (partial burst), and TX power (full burst).
For 802.11b signals, you can observe a similar time savings by performing composite measurements. With this signal type, important measurements include EVM, power, power ramp-up time, and power ramp-down time. Again, composite measurements are one way to improve device test times because you can use them to perform all measurements simultaneously. The results shown in Table 5 are from an NI PXIe-8106 dual-core controller running LabVIEW 8.6.1. The EVM measurement is performed over 1000 chips, and the gated power measurement is computed over a 100 µs time interval.
Table 5. Measurement Time of Composite versus Single Measurements for 802.11b
Again, Table 5 shows that it is more efficient to perform all measurements in parallel than to perform each individually. For a 11 Mbps CCK burst, an EVM, a TXP, and ramp-up/ramp-down measurements take 126 ms when performed sequentially but only 64 ms when performed in parallel.
4. Trade-Off 4 – Measurement Span versus Measurement Time
A fourth trade-off to consider when performing WLAN spectrum measurements is the relationship between measurement time and measurement span. While the IEEE 802.11 standard defines a 60 MHz mask for 802.11a/g signals and a 66 MHz mask for 802.11b signals, there are several instances when a custom span is required. For example, a validation engineer might want to evaluate a 100 MHz span to check for spurs farther away from the modulated signal. Moreover, an engineer configuring a production test system might use only a 44 MHz span for an 802.11b signal to save on measurement time.
In both digital IF analyzers and traditional swept-tune analyzers, wider span measurements require longer measurement times. With traditional swept-tune analyzers, there is a nearly linear relationship between measurement time and span. In this case, a 100 kHz RBW filter is swept through the desired span, and the measurement duration is linearly correlated with the span. With vector signal analyzers (such as the NI PXI-5661 and NI PXIe-5663), however, the results are slightly different. For spectrum measurements that are narrower than the instantaneous bandwidth of the vector signal analyzer, the RF front end of the instrument is not required to re-tune to complete the measurement.
For example, the NI PXIe-5663 RF vector signal analyzer offers an instantaneous bandwidth of 50 MHz. Thus, you can perform spectrum measurements that are less than 50 MHz in span without paying the time penalty associated with ret-tuning the front end of the instrument. In figures 5, observe that you can use an NI PXIe-8106 controller to perform spectrum measurements anywhere from 3 to 12.5 ms, depending on span.
Figure 5. WLAN 802.11a/g Mask versus Span for an NI PXIe-8106 Controller (NI RFSA 2.2 or Later)
However, for spans between 50 and 100 MHz, you need to retune the RF front end of the analyzer exactly once. Thus, along with the additional signal processing required by the CPU, re-tuning the analyzer front end contributes to the overall measurement time. Figure 5 shows that a 66 MHz span (the full 802.11a/g mask) takes approximately 12.5 ms. In this case, the additional time is due to the settling time of the LO (local oscillator) and not an increase in processing time.
Note that similar to EVM measurements, you must also consider the relationship between measurement time and the number of averages. Engineers often perform several averages because averaging provides a reasonable depiction of the noise floor. In Figure 6, observe the difference between the spectrum mask measurements (66 MHz span) of one average versus 100 averages.
Figure 6. Averaging produces less measurement uncertainty for the spectrum mask measurement.
Thus, both measurement bandwidth and the number of averages affect the overall speed of spectrum mask measurements. In general, the measurement bandwidth significantly affects measurement time only when the RF front end is required to tune. The number of averages, on the other hand, has a linear correlation to measurement time.
For example, consider the 802.11b spectrum mask measurements (44 MHz span), one of the more processor-intensive measurements. Figure 7 shows a nearly linear relationship between measurement time and the number of averages.
Figure 7. 802.11b Spectrum Mask Time versus the Number of Averages for Various CPUs
Moreover, CPU measurement time is highly dependent on the particular CPU. In this case, CPUs with better computational capabilities, such as the NI PXIe-8106, are able to perform this measurement significantly faster than others.
5. Trade-Off 5 – Effect of CPU on Measurement Time
A fifth factor that significantly influences the measurement time of WLAN signals is the specific CPU used in the measurement system. The CPU is one of the fundamental cores of a software-defined PXI measurement system. Especially for RF measurements, CPU performance is often the single most significant factor preventing faster measurement performance. Fortunately, you can use modern multicore CPUs the WLAN Analysis Toolkit to deliver industry-leading measurement results.
While actual system performance can depend on a variety of factors such as memory available and other applications running in the background, a strong correlation exists between the CPU performance and measurement time for automated test systems. Thus, the following table and figures are based on a benchmark using the PXI controllers listed in Table 6.
Table 6. Key Specifications of Various PXI Express Controllers
Several CPU characteristics can affect overall measurement speed. Some of the most significant of these include the number of processing cores, CPU clock speed, front side bus, L2 cache size, and system memory.
Figure 8, which illustrates the relationship between time and the burst data rate, shows the effect of CPU on EVM measurement time. You can see that faster dual-core controllers such as the NI PXIe-8106 perform EVM measurements faster across all data rates.
Figure 8. Measurement time improves with faster CPUs.
Although the NI PXIe-8106 is faster across all data rates, note that it does not have the fastest clock rate of all controllers used in the experiment. While the NI PXIe-8130 – which uses an AMD CPU – has a faster clock rate than the NI PXIe-8106, the results suggest that other factors such as a smaller L2 cache size also influence computation speed. The Intel Core 2 Duo T7400 CPU used in the NI PXIe-8106 has the largest L2 cache size (4 MB) of any CPU in this sample set.
6. Conclusion
As you have seen from the measurement results in these tables and figures, a variety of different factors can affect the overall measurement time of WLAN signals. Thus, to optimize a measurement system for speed, you need to carefully consider settings such as the number of averages, symbols to measure, and measurement span (spectrum). Moreover, while you can adjust many measurement settings to improve measurement time, these settings can occasionally require you to trade off repeatability, accuracy, or completeness of the measurement. Thus, the easiest way to improve test throughput without sacrificing measurement quality is to always use the fastest CPU available. Fortunately, the ability to choose your CPU is one of the core benefits of software-defined PXI test systems. In addition to enabling significantly faster measurements than traditional boxes, PXI systems are highly customizable. As a result, you have the flexibility to upgrade your processor in the future to achieve even faster measurement times.
7. Resources
View additional details on WLAN test systems
View pricing and specs for a pre-configured WLAN Test System
Configuring Software-Defined WLAN Test Systems
Request your free RF and Microwave Reference Poster
The mark LabWindows is used under a license from Microsoft Corporation. Windows is a registered trademark of Microsoft Corporation in the United States and other countries.
