Stream to Disk Using Win32 File IO

Publish Date: Aug 23, 2007 | 6 Ratings | 3.83 out of 5 | Print | 3 Customer Reviews | Submit your review

Overview

The LabVIEW VIs below provide access to non-buffered file I/O in Windows. This is useful for sequential reads and writes of large amounts of data at high data rates. Non-buffered file I/O disables the Windows file cache, so data is written directly to or read directly from the file I/O driver for the hard disk drive or RAID controller.

The Read VIs in this directory take an incoming array as input; the data read from disk is placed in the input array provided.  This leads to much higher performance, since the array can be preallocated ahead of time.  The Read Binary File VI in LabVIEW performs at much lower rates

The write VIs in this directory are there only for completeness; you should be able to use the Win32 Open File VI to open the file in non-buffered mode and use the LabVIEW Write Binary File VI thereafter.

These VIs are compiled in LabVIEW 8.0, although they should work in earlier versions of LabVIEW if saved for that version.

Note that the Win32 Open File VI outputs a LabVIEW file refnum as well as a Windows file handle.  This allows you to call LabVIEW file I/O VIs such as Get File Size or Set File Position along with the Win32 Read File VIs.

Benchmark VIs that help you test the maximum read and write speeds for your hard drive or RAID configuration are located in the FileIOBenchmarkVIs directory at the same level as this Win32FileIOVIs directory.

It is important to note that data must be written or read in quantities that are a multiple of the sector size (512 bytes). Data must be aligned in memory on boundaries required by the IDE, SATA, or RAID controller.  The memory alignment requirement is typically 2 bytes, although nVIDIA onboard RAID controllers require 4 bytes and Adaptec RAID controllers require 8 byte alignment.  Most data arrays allocated by modern compilers are 8-byte aligned, so this is usually not a problem.

Please copy and extract the file below to the C:\Program Files\National Instruments\LabVIEW 8.2\user.lib folder to see Win32 File IO Pallette within "User Libraries" of the functions pallette.

Back to Top

Customer Reviews
3 Reviews | Submit your review

Improving throughput to filesystem  - Mar 25, 2011

One thing we have done is to pre-allocate our data files. This means that when the data is written, the file subsystem of the OS doesn't have to search for and allocate space on the disk -- it's already there. We pre-allocate to a file that will be larger than what we will write, and then when the file is closed, the EOF is cut back to the last position in the file (at the end of the last record). This also helps prevent file system fragmentation. So far as we have seen, when a file is pre-allocated using the set EOF method, it is contiguous. The caveat is (if the files we generate are to remain contiguous) to <b><i>never</i></b> write past the end of the pre-allocated file.

Great!  - Nov 21, 2007

Worked great for me....I went from 7-8 MB/s to 47 Mb/s. Perhaps it is system / hardware dependent? Or perhaps in control panel / devices / drives you had write caching disabled already?

  - Nov 5, 2007

Well written but didn't seem to improve I/O much. 35 Mb/s with the LV method and 39Mb/s with the win32 method, both on my Dell Latitude D820 notebook/laptop

Bookmark & Share


Downloads

Attachments:

win32fileiovis.zip


Ratings

Rate this document

Answered Your Question?
Yes No

Submit