In-Vehicle Datalogger Reference Application for CompactRIO

Publish Date: Jan 30, 2019 | 15 Ratings | 3.67 out of 5 | Print | 2 Customer Reviews | Submit your review


This reference application presents a software solution for a stand-alone embedded datalogger using Compact RIO hardware.  The feature set of this application is tailored for in-vehicle logging applications, although it is also suitable for other types of logging. Note that this reference application is a configurable example, not a turn-key application. Most uses of this reference application require some modification of the code to reflect the individual hardware used.

See the Parts List for this application now

Table of Contents


This reference design library was created by the NI Systems Engineering group. 

You can give us feedback by posting questions and comments through the CompactRIO In-Vehicle Datalogger Reference Application discussion thread


Features of this reference application include:

•         Support for multiple scanning rates

•         Support for logging of CAN channels with an appropriate CAN database

•         Support for TDMS files

•         Controller logging indicators and override via LEDs and switches

•         PC host monitor mode and logging override


Component Requirements

In addition to the hardware and software requirements, listed below, this reference application uses the following independent components, please install both before opening the application:

Simple TCP/IP Messaging (STM) Component

VI-Based API for Writing TDMS Files

Advanced CAN Frame to Channel Conversion Example (install in user.lib)


The following diagram provides a detailed look at the operations performed by the datalogger VIs as well as the flow of data throughout the datalogger.


Communication Mechanisms

The following sections describe the communication mechanisms used by the datalogger.

Data Flow

Data flow is the normal transfer of data between block diagram objects by using wires.  Data flow is generally used for transferring data between the functions within a loop.


Queues transfer buffered data between loops.  

Note that the use of Real-Time FIFOs is often encouraged over the use of queues for Real-Time code.  This is because queues allocate memory at run-time, an operation which is non-deterministic.  However, in order to maintain determinism, Real-Time FIFOs enforce a number of requirements which limit the types and size of elements.  To remain flexible, the in-vehicle datalogger must be capable of accepting data packets to log to disk from a variety of sources and at a variety of rates.  This necessitates a flexible inter-loop data transfer mechanism.  In addition, the Real-Time segment of the datalogger does not require a great deal of determinism, since the FPGA performs all deterministic operations and does not have any timing dependencies on the Real-Time code.  Therefore, the in-vehicle datalogger intentionally implements inter-loop communication using queues rather than FIFOs.

Direct Memory Access (DMA) is used to transfer buffered data between the Real-Time operating system and the FPGA on a CompactRIO.  DMA data transfers allow the Real-Time operating system to read all data points while still allowing the FPGA to execute deterministically without a timing dependency upon the Real-Time code.

The in-vehicle datalogger uses three FPGA->RT DMA channels: one to transfer High-Speed Data, one to transfer Low-Speed data, and one to transfer CAN messages and other event-based data.

FPGA Control I/O 

Controls and indicators on an FPGA VI can be read and written either from within the FPGA code using local variables or from a Real-Time VI using the Read/Write Control function.

The in-vehicle datalogger uses FPGA Control I/O to scan and set data items that do not require buffers, such as status and configuration information.


The Simple TCP Messaging (STM) component simplifies network communication by establishing a message based protocol for transferring data.

The in-vehicle datalogger uses STM messages for all direct communication between the CompactRIO and Windows host.


File Transfer Protocol (FTP) provides a safe protocol for transferring files across a network.

The Windows host provided for the in-vehicle datalogger uses FTP VIs from the NI LabVIEW Internet Toolkit to programmatically retrieve data and error files from the target.  If you do not have the Internet Toolkit, you can replace these VIs with an alternate FTP implementation, or disable this function of the host and transfer files manually using an FTP client.

FPGA Calibration VI Details

The FPGA Calibration VI retrieves calibration constants stored on C-Series hardware modules.  This VI is separated from the main FPGA acquisition code because calibration values only need to be retrieved once per acquisition.  Separating the calibration code into a separate VI uses fewer gates for the main VI.


The VI reads calibration constants from each acquisition module.  You must modify these sections of code to read calibration constants from the modules you configure in your system.

Calibration data is bundled into weight/offset clusters and then built into arrays for transfer.  The arrays must have an entry for each channel.  Channels which do not require calibration, such as digital channels, are represented by a cluster of two zero values.  Front panel indicators transfer the data using FPGA Control I/O.  Note that calibration code is not provided for event based data. If your event based data requires calibration constants, you should implement a separate indicator to transfer the calibration data.

FPGA Acquisition VI Details

The FPGA Acquisition VI waits for a trigger condition, acquires data from C Series modules until a stop trigger or condition is detected, and transfers the data to the Real-Time controller.

 System Manager Loop

The System Manager Loop checks for stop triggers or a stop command from the Real-Time controller and transfers stop messages to the other loops in the system.

The datalogger allows you to configure both start and stop triggers.  You must replace the FPGA I/O nodes which detect external triggers with nodes linked to your own trigger channels.  If your system does not require external triggers, you may replace the nodes with constant values and avoid using the digital or analog trigger types.

 High-Speed Loop

The High-Speed Loop handles acquisition from timed acquisition modules, such as the 923x series.  Alternately, this loop can scan other modules at a separate sampling rate than the Low-Speed loop.

The loop reads a single data sample from each high-speed channel.  Replace the I/O node with one or more I/O nodes configured for your high-speed channels.  You can read data from timed modules, normal modules, or both types of module from within this loop.  If timed modules are present, the acquisition rate is controlled by the HS Scan Rate control and the HS Scan Period control is ignored.  Otherwise, the HS Scan Period determines the rate and the HS Scan Rate is ignored.

All data is transferred in raw format via a DMA buffer.  You may need to modify the size of the memory buffer in the project to decrease the chance of overflows or preserve resources on the FPGA.  By default, the high-speed DMA is allocated a much larger buffer than the low-speed DMA to allow for higher scanning rates in the high-speed loop.

Each loop checks to see if the loop executed in the configured time period (overrun), and if there was sufficient room in the DMA buffer for the data points (overflow).  The loop latches any errors so that subsequent iterations will not erase the error indication before it can be read.

 Low-Speed Loop

The Low-Speed Loop handles acquisition from other modules, at a different sampling rate than the high-speed loop.

The loop reads a single data sample from each low-speed channel.  Replace the I/O node with one or more I/O nodes configured for your low-speed channels.  You can read data from any module that does not use internal timing.  The LS Scan Period determines the acquisition rate.


This loop also uses the DMA Write and Status Check functionality.

 Event Loop

The Event Loop acquires messages from CAN channels and can optionally be used to acquire other event-based data.

The loop reads messages from a CAN channel.  Link the I/O node to the CAN channel you are reading.  Note that the current code is set to read only a single CAN channel.  You can use the timeout case in this loop to handle other event-based channels.  Configure the CAN timeout through the advanced module properties in the project explorer.  If multiple CAN channels are required, you will need to modify the code to handle the timeout from each CAN channel individually.

The Event loop transfers data over DMA by constructing packets.  Each packet contains an event index, which corresponds to a configured event channel, as well the size of the event data, and the data itself.  You may need to modify the size of the memory buffer in the project to decrease the chance of overflows or preserve resources on the FPGA. 

The Event loop checks to see if there was sufficient room in the DMA buffer for the event (overflow).  The Event loop does not check for overrun because there is not an expected loop period.  The loop latches any errors so that subsequent iterations will not erase the error indication before it can be read.

RT Controller VI Details

The Real-Time Controller reads the datalogger configuration from a file, acquires data from the FPGA, stores it to disk, communicates with the Windows Host, and interacts with the user via switches and LEDs.


The FPGA Comm loop interfaces with the FPGA and transmits data to and from the rest of the system.

In its Run state, the FPGA Comm loop reads the three DMA channels which transfer measurement data and events from the FPGA.  The loop gathers data read from the DMA channels into packets and transfers it to the File I/O loop using a queue.  The FPGA Comm loop also reads status information from FPGA Front Panel controls.   Status information is transferred to other loops using the Status Manager.

In its Get Calibration state, the FPGA Comm loop reads calibration data from the FPGA Calibration VI using FPGA Front Panel controls.  The loop stores calibration data in the Channel Manager for use by the File I/O loop.

A Status Manager functional global variable stores status information about the system.  Multiple loops access this storage location to read or modify the system status.

A Channel Manager functional global variable stores a list of channels and information about each channel.  The channel list is initialized with data from the  Real-Time Controller's configuration file.  Calibration information is stored by the FPGA Comm Loop.  The File I/O Loop uses the stored information to access channel information and convert raw data to engineering units.

The Real-Time Error Log (RTEL) component provides a simple method for logging errors that takes into account Real-Time performance considerations such as memory allocation.  Although optimized, the component is not safe to use from a truly deterministic loop because it accesses a file on disk.  The RTEL stores errors in a simple ASCII format with a fixed size per line and a definable maximum file size.  After reaching the maximum file size, the RTEL overwrites the oldest errors with new ones.  Note that the errors in a RTEL log file are not necessarily in chronological order for this reason.  The host VI includes a utility to convert RTEL logs to chronological order.

File I/O Loop

The File I/O loop receives raw data from the FPGA Comm Loop, categorizes the data by channel and group, converts the data to engineering units, and logs the data to disk.

The File I/O loop can briefly pause logging to allow the safe removal of storage devices.  Logging can be paused from a Windows host, or using a switch on the CompactRIO.  Note that when logging is paused, data is still acquired and buffered if the system is triggered.  Leaving logging paused for an extended period of time will lead to a buffer overflow.  The amount of time you can safely pause a system depends upon the acquisition rates involved.

The Channel Based File I/O (CBFIO) component is a high-level API for writing channel based data to a file.  Currently, this API stores files in TDMS format, using a VI-based TDMS Implementation.  TDMS provides a flexible file format that can be written from any operating system, and directly accessed by analysis programs, but does not necessarily provide the fastest streaming rates available.  The CBFIO component is designed with the ability to write multiple file types in mind, and additional file formats may be supported in the future. 


This loop also uses the Status Manager, Channel Manager, and RTEL functionality.

Switch/LED Control Loop

The Switch/LED Control loop updates LEDs with values from the Status Manager, and sends commands to the File I/O loop via a queue when switches are changed.

The Switch/LED Control loop updates the USER1 LED on the CompactRIO controller.  The value of the USER1 LED displays whether logging is enabled.  You can use this LED to determine when it is safe to remove a storage device after disabling logging.  You can also alter the behavior of this VI to display different information to the LED.

Note that the system also updates the FPGA LED which is provided on some CompactRIO systems, but the updates are performed from the FPGA code.  The FPGA LED displays whether data is currently being acquired.

The Switch/LED Control loop monitors the state of the USER1 switch on the CompactRIO controller.  When the switch is in the ON position, the Loop sends a command to the File I/O Loop via the logging queue to pause logging.  When the switch is in the OFF position, logging is enabled.  You can modify the behavior of this VI to change the function of the USER1 switch.  For example, you could make the USER1 switch trigger the acquisition start and stop rather than the log pause and resume.


This loop also uses the Status Manager and RTEL functionality.

Host Communication Loop

The host communication loop accepts connections from an optional Windows host machine.  When a connection exists, the loop updates the host machine with status information and allows the host machine to override the standard behavior of the logger.

Direct communication with the host uses the Simple TCP Messaging (STM) protocol.  The host can send messages to reset the system, pause logging, or change the monitor settings.  The target sends messages to transmit status information and monitored data to the host.


This loop also uses the Status Manager functionality.  This loop does not use the RTEL because errors in host communication are expected as connections are lost and reestablished.  In addition, the host is an auxiliary element of the system, and therefore error log space is reserved for more critical system behavior.  If you are using the datalogger in a system where the host is an essential tool, you can add the RTEL to the Host Communication loop and ignore the specific errors associated with lost connections. 

Windows Host VI Details

The Windows Host VI provides a user interface for monitoring and managing the datalogger, as well as viewing acquired data.

Event Handler Loop

This loop responds to events from the user interface, and reads STM commands in a timeout event.

The Host uses FTP to transfer data files and error logs from the target.

The Host uses STM to send pause, reset, and monitor settings messages.  The host receives monitor data and system status messages.

The host allows the user to configure the target network location, file locations on the target, local locations to store transferred files, and optionally, an acquisition channel to monitor.

The host allows you to override the target's normal operation by pausing data logging or resetting the target.  Note that some caveats apply to pausing the logging.

The host displays the system status, received from STM messages.  This includes whether an acquisition is currently running, and whether logging is currently paused.  It also indicates common errors that may occur on the target.  

An FPGA Overflow indicates that the DMA buffer is not large enough to store the transferred data, or the FPGA Comm Loop is not running fast enough to empty the buffer.

An FPGA Overrun error indicates that the acquisition loop cannot run at the desired sample rate.  Note that each FPGA I/O node takes a minimum amount of clock cycles based upon the channels it reads.

An RT Overflow error indicates that data could not be written to disk fast enough to empty the memory buffer between the FPGA Comm Loop and the File I/O Loop.  This indicates that the total system throughput is too high for the current architecture or file format to support.

The Host can monitor live data updates on a single channel.  This is primarily useful for debugging or monitoring purposes, and may be unavailable if there is not sufficient processor bandwidth on the target.  The Host also generates pop-up dialogs to display the contents of data files and error logs when they are successfully transferred.

Parts List

See the Parts List for this application now

Back to Top

Customer Reviews
2 Reviews | Submit your review

Missing VI added  - Feb 5, 2008

Missing VI added. The missing is now in the zip, and a link to get the G TDMS VIs is now on the page

missing vis  - Jan 17, 2008

missing vis... my computer/ and _G TDMS 8x.llb

Bookmark & Share





Rate this document

Answered Your Question?
Yes No