Building a Transient Recorder System for the CERN Large Hadron Collider

Odd Oyvind Andreassen, CERN

"CompactRIO was an ideal solution because it was the only modular embedded control and monitoring system on the market where we can have full control of the source code that defines the system functionality. "

- Odd Oyvind Andreassen, CERN

The Challenge:

Finding a software-defined modular platform for a transient recorder system for the CERN Large Hadron Collider (LHC) that includes a continuous measurement system with 10,000 samples/s, conducts real-time analysis of more than 100 channels on several data stations, synchronizes all data stations, properly buffer and store the correct data when a transient is spotted in the input signals, and to collect data on two servers in different locations so we can ensure redundancy, analyze elusive transients, and learn how to mitigate them.

The Solution:

Using the NI CompactRIO platform, NI LabVIEW Touch Panel Module, NI LabVIEW system design software, NI LabVIEW Real-Time Module, NI LabVIEW FPGA Module, and an advanced, event-based software architecture to build a flexible transient recorder system that is easy to maintain.

Author(s):

Bjarke Dahl-Madsen - CIM Industrial Systems A/S
Odd Oyvind Andreassen - CERN

 

CERN: The European Organization for Nuclear Research

Physicists at CERN are using some of the world’s most powerful particle accelerators to learn more about what the universe is made of and what its origins are. Physicists and engineers use large, complex scientific instruments to study the fundamental particles. They make the particles collide at close to the speed of light to get information about how the particles interact and gain insight into the fundamental laws of nature. CERN, founded in 1954, is located at the French-Swiss border near Geneva. It was one of Europe’s first joint ventures and now has 21 member states. 

 

 

The CERN LHC is the world’s largest measurement system and requires an enormous amount of electricity to operate. About 10,000 superconducting magnets need to be cooled down to 1.9 K (-271.3 °C or -456 °F), which is colder than outer space.

 

In the early days, a substation on the Swiss side of the campus provided enough electricity to meet CERN’s needs. However, in the 1970s a line was installed to connect a new substation on the French side of CERN. This substation now powers all of CERN and the Swiss substation is maintained as a partial backup. Seven substations, some in France and some in Switzerland, handle the electrical power transfer.

 

 

 

A Transient Recorder System Needed for Safe Operation of the LHC

The LHC requires good electrical transfer quality. There can be a risk of external factors, such as thunderstorms, or energy supplier induced transients, affecting the measurement and control systems. When a transient occurs on the electricity network, the accelerator is usually forced to stop, which is frustrating and expensive. The transient recorder system offers engineers a better understanding of when, how, and where the transients occur. With this knowledge, they can find a way to mitigate the transients on either the supply- or control-side of the LHC. This helps minimize unforseen shutdowns and gives us as much uptime as possible. 

The transient recorder system needed to:

  • Perform real-time recording and analysis of all transients
  • Include data acquisition rates of 10 kHz/c across more than 100 channels
  • Record several seconds of data
 
  • Communicate between different operating systems such as Linux, Windows, and real time  

We tried to find different vendors for this kind of system, but in the end, we realized that the NI CompactRIO platform was the only system on the market that gave us full control of the software. Other systems were more like “black boxes” in which the vendor defined the system. We wanted to define the system ourselves so we could modify it for our future needs. We were also happy that the CompactRIO platform is fully programmable with NI LabVIEW. LabVIEW is used on a daily basis at CERN, and we have more than 500 active LabVIEW developers. We can use LabVIEW to program the GUI, the real-time processor, and the FPGA.

 

 

 

System Architecture for the Transient Recorder System

Because of the complexity of the system, we used CIM Industrial Systems A/S, an NI Alliance Partner in Denmark, to design the architecture and provide a solution that we could easily update. CIM was an ideal partner for this project, because of its expertise using LabVIEW and CompactRIO systems. The company already had multiple LabVIEW modules available to reuse within this project, which saved a lot of development time. CIM provided a system that will be easy to implement, maintain and modify, for many years to come.

We based the transient recorder system architecture on an event-based architecture similar to the common actor framework. This design was used for creating LabVIEW applications that consist of multiple, independent tasks that need to communicate with each other. We designed the chosen framework to address common development scenarios that can lead to significant duplication of code when extending functionality or adding additional processes. Furthermore, several developers could easily work simultaneously on the design and coding tasks.

CIM used the LabVIEW integration with the syslog protocol to ensure a good debugging environment for this complex multinode system. This means a single engineer can access everything that happens on the system from any work station. It also provides for a real-time overview of how analysis generated triggers propagate throughout the system, how data is extracted from the buffer, and how data is stored and requested from the data servers. 

 

 

A Solution for a Modular and Flexible Transient Recorder System for the CERN LHC

We built a modular and flexible transient recorder system for the CERN LHC using LabVIEW system design software and the CompactRIO embedded platform. CERN engineers can easily analyze the large amount of data collected, which helps the electrical engineers to understand the origin of these transients and minimise downtime for the LHC.

The system’s flexibility empowers us to further expand the real-time analysis conducted with the system. With the new generation of CompactRIO hardware with multiprocessors, we can expand the system to analyze even more channels and with a higher sampling rate, which means we can catch even more elusive “nth-order” phenomena.

 

Author Information:

Odd Oyvind Andreassen
CERN
odd.oyvind.andreassen@cern.ch

Figure 1. The CERN LHC in the Tunnels Beneath France and Switzerland (Image Courtesy of CERN)
Figure 2. CERN’s Power Substation in Prévessin, France (Image Courtesy of CERN)
Figure 3. CERN LHC Aerial View (Image Courtesy of CERN)
Figure 4. System Architecture