Use the Historical Data Viewer to view the data stored in any Citadel database. Select My System»Historical Data in the Configuration tree of NI Measurement & Automation Explorer (MAX) to open the Historical Data Viewer.
Citadel organizes data into a three-tiered hierarchy containing the originating computer name, the process name, and the trace name. Figure 1 displays a graphical overview of a typical Citadel database in the Historical Data Viewer.
Figure 1: Citadel Database Structure
The Historical Data Viewer also provides access to multiple databases on the same computer and to remote databases.
In addition to trace views, Citadel provides waveform hierarchies containing data logged from VI Logger and dataset hierarchies created using the DataSet Marking I/O Server in the DSC Module.
Traces and Subtraces
Citadel organizes databases into traces or datasets. Traces are uniquely named and identify a group of sub-traces that comprise a historical record of trace values. Subtraces are composed of data runs with constant resolution and logging parameters.
A subtrace is an arbitrary data stream containing a single type of data with associated meta-data. Subtraces are indexed by time, so you can identify a time position within a subtrace without reading through the entire stream. Because data points are indexed by time, all data in a subtrace data stream must have non-decreasing timestamps. If a back-in-time event occurs, or if the data type changes, Citadel must create a new subtrace. Changing the data type involves changing the base data type, such as double or Boolean, or changing the format with which data is serialized to disk, such as changing the compression algorithm by changing the value or time resolution, changing the type of variable being logged to a variant subtrace, or even changing the attributes that are set on the variant. NI recommends that you do not set attributes on variants logged to Citadel.
Though Citadel provides no direct interface to work with individual subtraces, knowledge of subtraces is important because subtraces can have a significant impact on database performance. In particular, if a trace contains many subtraces, the trace performs less well than a trace with fewer subtraces. In typical use cases Citadel does not need to create multiple subtraces.
If the Citadel service or any local Citadel client, reader, or writer fails to shut down cleanly while accessing a database, Citadel re-indexes the database upon restart. When Citadel re-indexes a database, a subtrace may be damaged by the failure that triggered the re-indexing. Therefore, when Citadel re-indexes a database, all existing subtraces are closed permanently and new data is logged into a fresh subtrace.
The following events can trigger the creation of a new subtrace:
· The trace data type changed.
· You log a “back-in-time” value.
· The logging properties, such as logging resolution, changed.
· The Citadel service terminated abnormally, such as during a power loss or system crash.
Subtraces consist of data runs of up to 100 points. For example, data in analog traces is compressed based on delta values and the logging resolution. After every 100 points, an absolute measurement value is logged, which resets the compression algorithm and creates a new subtrace. This behavior prevents accumulation of errors and reinforces data integrity within the database.
Datasets are an alternative grouping of data that associate a “run” with a group of traces and time ranges. Datasets can be created using VI Logger and the DataSet Marking I/O Server in the LabVIEW DSC Module.
Citadel associates a variety of properties with each trace in the database. Use the Historical Data Viewer to review these properties. Right-click a trace in the Historical Data Viewer and select Properties from the shortcut menu to display properties for the trace. Figure 2 displays an example of a trace Properties dialog box.
Figure 2: Trace PropertiesDialogBox
The Data type property indicates the most generic type associated with a trace. The most generic type determines the reader to use to retrieve the majority of the trace contents.
The data types that Citadel support, from most to least generic, are:
- Binary (LabVIEW String)
- Unicode String
- ANSI String
The startTime and endTime properties of a trace indicate the actual start and end times of the trace in Universal Coordinated Time (UTC) format. UTC avoids all conflicts associated with Daylight Savings Time or varying time zones. Although Citadel stores timestamps in UTC, the Historical Data Viewer displays times in the local time zone. The Historical Data Viewer appends the current time zone name to the end of the time string in the Value column of the Properties dialog box.
The lifespan property indicates how long trace data remains in the Citadel database before Citadel reuses storage space taken up by old trace data. For instance, if you configure a trace with a lifespan of 10, only the ten most recent days' worth of data remain in the database. After ten days, Citadel overwrites the oldest data. The lifespan of a trace is defined by the application that writes data to the database and can be configured only for each process, not for each trace. A lifespan of 0indicates an infinite lifespan.
Setting a lifespan indicates specifies that Citadel can delete data older than the lifespan in order to save space. However, the database does not guarantee the deletion of expired data on any schedule. Lifespanning occurs only when the process that originally logged the data actively logs new data to the database. For example, if you copy data to a CD and, 15 years later, reattach your database to retrieved data, the data does not disappear due to the lifespanner.
The pages property indicates the number of pages the trace is using and provides a rough estimate of how much disk space the trace uses. Each page takes up 4KB of space.
The subtraces property indicates how many subtraces are in the trace.
VI Logger Data
VI Logger stores waveform data to Citadel in groups of runs that belong to VI Logger tasks. Each run corresponds to a single acquisition of data. A VI Logger run consists of between one and three subtraces: a waveform subtrace containing data from hardware, a waveform subtrace containing calculated data that the VI Logger application generates, and a subtrace containing VI Logger event data. VI Logger subtraces contain raw waveform data stored as arrays of uncompressed values. Waveform subtraces do not contain serialized timestamps because the waveform time stamp can be calculated from its startTime and deltaT attributes. VI Logger uses uncompressed subtraces to maximize throughput to disk, at the cost of greatly increased disk and network usage. The VI Logger event subtrace uses the same value or time stamp format as single-point traces to log arbitrary-length Boolean arrays with compression.
Supported Data Types
Though optimized for numerical data logging, Citadel supports the following data types:
· Analog—Citadel compresses analog values based on the specified logging resolution.
· Integer and bit-array—Citadel does not compress discrete integers. BitArray subtraces are stored as 32-bit (DWORD) unsigned integers.
· String—Citadel supports three string types: LabVIEW strings, null-terminated strings, and Unicode strings.
· Variant—Citadel stores raw variant data but optimizes storage by storing the metadata associated with a variant only once instead of with each logged value.
· Waveform—You can use VI Logger and DIAdem to interact with waveform data.
Database Size Considerations
Citadel does not impose a size limitation on databases. However, NI recommends you take the following performance considerations into account when planning a data logging system.
· Citadel database performance degrades slightly as the database grows. This behavior is due primarily to the physical limitations of hard disks and to the limitations inherent in the NTFS and FAT32 file systems.
· Archiving large databases takes a significantly longer time than archiving smaller databases.
· Database integrity is not related to database size.
The size of a large database typically falls in the range of 5GB to10GB. The size of a medium-sized database typically is 2GB to 5GB, and the size of a small database is less than 2GB. In many applications, database lifespans range from 1 to 15 years.
Properly managing database size can improve your ability to maintain a database over its expected life time. NI suggests you consider the following tips to plan a healthy database system.
· Use an appropriate logging resolution and logging deadband.
· Use an appropriate lifespan setting when configuring a process.
· Plan for regular archiving operations to reduce the amount of data stored in the operational database. To achieve maximum performance, you might want to store only the last 1GB worth of data in the operational database. The actual amount of data you decide to keep in your operational database depends on the application.
Historical Alarm and Event Data
Each distinct piece of alarm and event data is composed of several fields of data of varying types. Although Citadel can store numeric data efficiently, Citadel is not optimized for storing record-based data. Storing alarm and event data in the primary Citadel format can lower efficiency and provide a less than optimal mechanism for returning alarm data for viewing.
NI stores alarm and event data in an attached Microsoft SQL Server 2005 Express Edition (SQL Server Express or SSE) relational database. SSE is freely redistributable and therefore does not impact the pricing of the DSC Module. SSE also can be upgraded to Microsoft SQL Server if necessary. NI recommends that you treat the attached SSE database as a part of Citadel and access it via the API provided with the DSC Module.
The alarms and events portion of a Citadel database is limited to 4GB. The SQL Server Express database imposes this restriction. If you require more than 4GB alarm and events in a single Citadel database, you can purchase SQL Server 2005 Workgroup Edition from Microsoft, which removes the 4GB restriction.
Citadel stores databases in a designated group of files on the hard drive. A Citadel database typically resides in a folder unique to that database. You can store only one Citadel database in a folder. NI recommends that you avoid placing non-database related files in a folder containing a Citadel database. A typical database is comprised of a set of files similar to those in Figure 3.
Figure 3: Typical Citadel Database Files
The number of .cdpg and .cdib files varies with the amount of data in a database. The nodetree.*, pid.cdih, and stridm.cdin files contain important information about the structure of the database. The mssql.* files contain historical alarm information. Citadel creates the mssql.*files the first time alarm or event data is written to the database.
Caution: Do not modify, move, or delete a database file while the database is attached. Doing so results in a database corruption. If you modify or delete a database file while the database is detached, you might not be able to reattach the database, and you might lose some or all of the data in the database. If you move or copy a detached database, move or copy all database files.
The .cdpg files contain trace data. Citadel stores data in a compressed format; therefore, you cannot read and extract data from these files directly. You must use the Citadel API in the DSC Module or the Historical Data Viewer to access trace data. Refer to the Citadel Operations section for more information about retrieving data from a Citadel database.
Each of the 1,024KB .cdpgfiles contains a set of 4KB pages. Each page contains data for a single subtrace. Citadel attempts to use all 4KB of space in a page before opening a new page. If a subtrace is terminated or a new subtrace is started before the end of a page, the remainder of the space in the page goes unused. Citadel reclaims the remaining unused space only if all trace data in that page is removed or deleted.
Compression of Numeric Data
Citadel uses a complex compression algorithm to store numeric data. Equation 1 approximates this compression algorithm.
where ∆n≥1 is the Citadel delta value, y is the set of recorded values, k is the set of logged values, and kn=0 = yn=0. The number of values, n, in a run always is less than or equal to 100.
Note: This section does not describe the full algorithm Citadel uses for compression. You do not need to understand this section to use or understand Citadel.
Data is typically acquired from a data source such as a data acquisition board, an analog input device, or a PLC. In most cases, measured values are passed through a series of deadbands before being sent to Citadel. Citadel then converts the values to deltas and compresses the values using the logging resolution you specify.
When queried for historical data, Citadel always returns a value equal to the measured value plus or minus the logging resolution at the time the data point was logged. The actual resolution of the returned data point might be higher than the specified logging resolution but is always within the logging resolution.
Table 1 provides sample delta calculations using Equation 1 and a logging resolution of 0.1, which retrieves only those values within ±0.1 of the actual recorded value.
Table 1: Example Citadel Delta Calculation
The logging resolution is a key parameter in how Citadel compresses numeric data. The logging resolution can be any real number but must be chosen in concert with the logging deadband. For example, if you specify a logging resolution of 10 units and a logging deadband of 1 unit, Citadel logs any change of at least 1 unit. However, due to compression and the logging resolution you set, Citadel cannot discern any value change under 10 in data retrieved from the database. To avoid this situation, always specify a logging resolution less than or equal to the logging deadband. Also specify the logging deadband as a power of ten unless you have an application-specific reason to do otherwise. Specifying a logging resolution of 0always logs data with full resolution.