Optimizing Streaming Performance

Publish Date: Sep 24, 2008 | 3 Ratings | 4.33 out of 5 | Print

Overview

There are numerous applications that require streaming large amounts of data at high speeds. Some examples include RF recording and playback as well as acoustic imaging. While modern rotating medium hard drives allow for high performance streaming, there are still limitations that might prevent hard drives from achieving optimal speeds. This article will discuss various strategies to maximize your streaming performance.

Table of Contents

  1. Introduction
  2. Fragmentation
  3. Cluster Size
  4. File Placement
  5. File I/O In LabVIEW
  6. Other Considerations
  7. Conclusion

1. Introduction

Figure 1 shows the main components of a mechanical hard drive. The components that mainly define the data streaming performance of a mechanical hard-drive are the platter, the actuator and the read/write head. Platter(s) consists of magnetic disks that spin at high speeds on the spindle and the actuator controls the position of the actuator arm. The head on the actuator arm is used to read and write data to the platter. For such mechanical hard-drives, operating systems such as Windows allocate file space from the outer rim inwards. This is also true for most RAID arrays.

Figure 1: Components of a mechanical hard-drive

Figure 2 illustrates how the read and write speeds drop off as the data is being read toward the inner diameter of the platter. Angular velocity is proportional to the radius; at larger radii the angular velocity will be higher resulting in higher data rates at the outer rim. Testing on a western digital hard drive showed performance at 62 MB/sec for a file at the outer rim of the disk, and falls to 36 MB/sec at the inner rim.

Figure 2: Change in read and write performance of a mechanical hard-drive as the data is written from the outermost rim to the innermost (X-Axis "0" -> Outermost Rim, "100" -> Innermost Rim)

Often times it is difficult to attain the specified read and write rates from a hard-drive or a RAID array. There are four main reasons that might cause this:

1.      Fragmented hard drive

2.      How the disk is formatted and cluster unit size

3.      File placement on disk

4.      LabVIEW file I/O used (Binary vs Win32)

5.      Maximizing Windows drive performance

Following sections explain these reasons in detail and also recommend some strategies on optimizing each category.

Back to Top

2. Fragmentation

A highly fragmented hard drive could result in lowered read and write speeds. Data is stored on the hard drive as clusters and when a file has to be written, the operating system stores the file in whichever free clusters it can find. This might create a file split up into clusters scattered throughout the drive platter. This requires the hard drive head to move to different locations on the disk to perform reads and writes. It takes time to write to these scattered free clusters, and when a file has to be read, it takes additional time for the head to get data from the clusters. Windows comes with a defragment utility that can be accessed by going to Start > Programs > Accessories > System Tools > Disk Defragmenter. This utility reallocates data from a particular fragmented file on to a contiguous block of clusters. Figure 3 shows an example of an highly fragmented hard-drive that will exhibit poor data streaming dates.

Figure 3: Example of a highly fragmented file

Back to Top

3. Cluster Size

How the disk is formatted also affects performance. When formatting a hard drive an NTFS logical partition, a master file table is created on the drive that contains all the attributes for the drive volume. One parameter important when working with large files is “Allocation unit size” which refers to the cluster size on the disk. When a file is saved to disk it usually takes up one or more clusters. When using RAID, larger cluster sizes result in better sequential I/O performance by reducing fragmentation on the drives and allowing larger files to be read in bigger chunks. Table 1 below gives an example of how the sequential read and write rates of an 8 drive RAID array are affected with different cluster sizes.

Cluster size

Write Speed

Read Speed

64 kilobyte

452 MB/s

445 MB/s

512 byte

402 MB/s

306 MB/s

Table 1: Effect of cluster size on data read and write performance

The formatting options for a logical partitions can be accessed by right-clicking on My Computer and choosing Manage. Go to Storage > Disk Management, right click on the hard drive and choose Format. Use the quick format option. Formatting the drive will erase ALL data and should be done right in the beginning of setting up the system. Ideally, a separate logical partition should be used for data storage than the one use to load the OS. Following figures illustrate this procedure.

Figure 4: Formatting a logical partition to set allocation size

Figure 5: Select the highest allocation unit size available for the parition

Another way to access formatting options is by right-clicking the relevant drive in Windows File Explorer, and choosing “Format”. However, there are less cluster size options here than in Disk Management.

Figure 6: Formatting partition and setting allocation unit size through Windows File Explorer

Back to Top

4. File Placement

File placement is also an important factor in hard disk streaming speed. If possible, a separate data drive formatted for a large cluster size is ideal. Having a drive dedicated to data allows you to reformat the partition at will when needed. If this is not possible, create a partition at the beginning of your drive for your data. This will allow all data to be written and read from the fastest part of the disk, resulting in maximum performance when accessing and writing your data.

Figure 7: Placement of a file in the specific logical partition affects sequential read/write performance

Back to Top

5. File I/O In LabVIEW

LabVIEW offers the ability to write and read data to disk in a binary file format. Using the binary format LabVIEW buffers and caches the data before writing. This limits streaming performance and it is difficult to write/read more than 100MB/s using the binary file format in LabVIEW. In order to overcome this limitation, National Instruments has a Win32 File I/O API available. LabVIEW 8.6 will have native support for the high speed Win32 File I/O API. If using LabVIEW 8.5 or earlier, the Win32 Vis can be downloaded here. The writes using the Win32 API are non-buffered and is useful for sequential reads and writes of large amounts of data at high data rates. It also disables Windows caching of data and uses queues to increase efficiency.

Table 2 below gives a comparison of the performance between binary and Win32 file I/O.

Drives/Controller

Type

Binary

Win32

1 (SATA)

Write

46

58

4 (ICH7)

Write

109

240

4 (Promise)

Write

144

243

8 (RocketRAID)

Read

58

438

12 (RocketRAID)

Read

38

723

12 (RocketRAID)

Write

68

617

Table 2: Sequential write performance difference between Binary File I/O & Win 32 File I/O

Back to Top

6. Other Considerations

Windows OS often times has background services running that access the hard-drive for file I/O.  These services are scheduled to run at specific time intervals and will cause a degradation of read/write streaming performance of the application. If there is a separate hard-drive or RAID partition that is allocated to data streaming then these background services can be disabled for this hard-drive. This will allow us to avoid unpredictable degradation in read/write performance.  Virtual memory, Recycle Bin & System Restore are few of these background services that can be turned off for specific drives to increase and maintain high data streaming rates. For optimum performance Write Caching must also be turned on for all drives. With Write Caching enabled, instead of writes being made immediately when demanded, data is written to the disk cache and then written to the disk when the cache is filled. This results in less number of writes and seeks. Figures 8 through 10 illustrate how to disable few of these background services.

Figure 8: Turning off System Restore can help in avoiding unpredictable streaming performance degradation

Figure 9: Disabling Recycle Bin

Figure 10: Turn on Write Caching to increase write performance

Back to Top

7. Conclusion

Disk performance is important when streaming data to and/or from disk. There are a number of ways to ensure that you are maximizing the potential of your hard drives. Keeping the drive defragmented is important as this ensures that the drive is not constantly writing or reading a file on different locations on the disk. Maximizing the cluster size will result in better sequential I/O performance. You should also allocate your data partition at the beginning of the drive to take advantage of the higher read and write speeds at that location. Using the high speed Win32 File I/O API that is natively supported in LabVIEW 8.6 and higher will also maximize data streaming performance. Implementing these steps will ensure maximum streaming performance from your National Instruments Data Storage hardware.

Back to Top

Bookmark & Share


Ratings

Rate this document

Answered Your Question?
Yes No

Submit