Saving $400M Through Condition-Based Maintenance​

Author(s):

​Gabi Berrios, Marketing Engineer, Cyth Systems
​David Patrick, Vice President, MSO Technologies​

 

 

​​Protecting Critical Grid Infrastructure

​Circuit breakers are critical parts of the electrical infrastructure, serving as crucial power delivery devices that continuously transfer electricity across the network. They instantaneously respond to current levels outside of predetermined thresholds, making them vital system management devices for protecting grid infrastructure and enabling operators to intervene when circuit breaker maintenance is necessary.

 

​Proactive Asset Monitoring

​On a mission to mitigate the risks associated with circuit breaker maintenance, one utility provider knew they needed more data insights to be able to reduce instances of unexpected service outages and circuit breaker events.

 

​Considering that circuit breakers in the electrical grid are expected to have a long operating life and high availability, it was critical for this utility provider to identify a solution that would enable them to leverage their service and maintenance crews optimally, while mitigating any risk to crews posed by unnecessary site visits.

 

​Their existing solution for addressing circuit breaker maintenance needs was to evaluate and diagnose issues during planned maintenance events or other on-site crew operations, resulting in the inability to capture the data from unanticipated breaker events during typical, day-to-day operations.

 

​Traditional maintenance approaches can result in operational blind spots, preventing comprehensive equipment performance tracking and leaving utility providers vulnerable to unexpected failures. A lack of data insights made it difficult to proactively stabilize the workload for service personnel, and challenging to recommend preventive maintenance services for their assets.

 

​To meet their grid availability and cost reduction goals, this utility provider needed a solution that would enable them to proactively manage their infrastructural assets.

 

​Building in Predictive Maintenance

​The types of deterioration that precede—and cause—most circuit breaker failures are detectable ahead of an actual failure. The ideal solution for safeguarding the utility provider’s infrastructure and workforce would be to install smart electrical grid technologies to measure and analyze all circuit breaker events.

 

​They needed a continuous asset health monitoring system that was reliable enough to continuously monitor the circuit breaker equipment and robust enough to handle the extreme weather and conditions throughout their extensive service area. With tens of thousands of circuit breakers across their service area, they needed to optimize the price point per installed monitoring system to make widespread condition monitoring economically viable.

 

​For this utility provider, evaluation and diagnosis of circuit breaker issues were limited to planned maintenance events or on-site operations, leaving them without visibility into normal and fault-condition breaker operations. This lack of insight made workload stabilization difficult for service personnel and prevented effective implementation of preventive maintenance.

 

​To maximize asset availability and optimally manage the workload of service crews, the utility provider needed to continuously monitor their circuit breakers for activity outside of normal operating thresholds. Data insights into circuit breaker activity would enable the utility provider to proactively prioritize and schedule maintenance based on the urgency and criticality of the issues identified.

 

​To ensure regulatory compliance, a circuit breaker must exhibit timely and adequate responses to electrical events. The performance of these circuit breakers is then provided to these regulatory bodies in the form of data that characterizes the response of a circuit breaker. This electrical utility’s regulatory requirements mandated annual testing of every circuit breaker in operation. Compliance with regulations and specifications is critical for public safety and ensuring reliable access to electricity.

 

​Considering that circuit breaker operations can occur at any time, compliance for a particular circuit breaker could be confirmed by observing how the breaker responded to an electrical event. With a solution for remotely and continuously logging data insights, this utility provider could dramatically reduce their cost of compliance without ever having to send a maintenance crew to the site.

​Seeking expert guidance, they turned to MSO Technologies for help delivering a solution capable of continuously monitoring their assets, providing data insights for predicting potential circuit breaker failures and ensuring regulatory compliance.

 

Unit	Description	​Hardware	Software
​MSO Circuit Breaker Analyzer (CBA)   Portable and ruggedized	Portable unit, taken on-site for testing during planned maintenance events	NI cRIO-9053	NI LabVIEW Real-Time and LabVIEW FPGA Modules   ​MSO RTOS daemon
MSO Circuit Breaker OnLine Monitor (CBOLM)   ​Permanently installed	Permanently installed units, event-triggered data monitoring system	NI sbRIO-9651 (SOM)   ​Cyth CircaFlex-580

 

​The utility provider wanted two form factors of the same system to provide the insights they needed while they scaled out and deployed permanent monitoring solutions to their tens of thousands of circuit breakers.

 

​One form factor would be a portable unit for assessing circuit breaker health during typical, reactive, or programmed maintenance events. The data gathered on-site would then immediately be used to identify issues and aid personnel in returning the asset to service. The second form factor would be a permanently installed system that would continuously monitor, capture breaker events, and publish the data to a remote server for immediate analysis and notification if warranted.

 

​Connecting Circuit Breakers to the IIoT

​MSO Technologies is an expert in delivering operational technologies and solutions to the energy industry that help seamlessly connect assets and business systems to the Industrial Internet of Things (IIoT).

 

​MSO turned to Cyth Systems for help because of their successes delivering highly responsive deployed measurement systems into extreme and challenging environments. Cyth designed and built the field signal interface for high-speed data acquisition. MSO software managed event trigger detection, data aggregation, and data publishing for subsequent analysis.

 

​To address all the circuit breaker event measurement use cases for the utility provider, MSO and Cyth defined requirements for two physical form factors of system running the same software stack.

 

1. ​Circuit Breaker Analyzer (CBA)

• Portable unit that utility maintenance teams take on-site to perform necessary measurements during scheduled maintenance events
• Cyth Lens Client™ software used on a laptop to configure event thresholds and consume pre-trigger and post-trigger data published by the CBA for immediate analysis, charting, and upload to the central Cyth Lens Enterprise™ Server
• ​NI CompactRIO hardware in a portable, ruggedized enclosure
• ​Standard connector enables connectivity with any brand of circuit breaker

 

 

 

2. ​Circuit Breaker OnLine Monitor (CBOLM)

• ​Permanently installed enterprise IIoT solution that interfaces with both the centralized server as well as clients to the server
• Continuously measures key circuit breaker health indicators and selectively logs data based on pre-determined thresholds
• The sbRIO platform was chosen for its compatibility with the RIO software stack, smaller physical and energy footprints, as well as cost point at volumes of scale and measurement performance in harsh and fluctuating outdoor environments
• ​Data logging is triggered by circuit breaker events
• Pre-trigger and post-trigger data is logged to Lens Enterprise™ Server
• Standard connector enables connectivity with any brand of circuit breaker

 

 

​The CBA and the CBOLM are both based on the NI RIO platform, enabling reuse of the same software stacks and codebases across both system types.

 

​MSO developed the solution for interfacing between the utility provider’s IT infrastructure and the NI RIO-based data acquisition and analysis systems. They created a daemon to run on the NI Linux Real-Time Operating System (RTOS) that is responsible for event threshold detection, securely logging event data and publishing to the remote server.

 

​Cyth developed a custom CircaFlex daughterboard to mate with the NI sbRIO-9651. The CircaFlex board enabled Cyth to incorporate the signal conditioning and connectivity necessary for ensuring data integrity and system reliability.

 

​The data acquired by the CBA and CBOLM is eventually pushed to a central database. The cRIO-9053 in the CBA unit acquires data that is then passed to a PC running MSO Lens Client™ software, which analyzes the data, charts the data for user review, and transfers the data to the central database.

 

​The sbRIO-9651 in the CBOLM continuously reads the signal trace inputs. When a circuit breaker event occurs, data logging is triggered by the sbRIO to ensure that the data immediately before and after the breaker event is captured and sent via MQTT to the central server. The published data is inserted into the central database, analyzed, and notifications to maintenance personnel are sent, if required. This data is then available for on-demand and remote viewing, so maintenance experts can retroactively view and analyze circuit breaker behavior and events to determine the optimal path to resolution.

 

​The compatibility of MSO’s code with the RIO hardware on both the CBA and CBOLM, combined with Cyth’s field-tested software architecture, helped accelerate the development of both solutions and will help ensure system sustainability long into the future.

 

​The implementation of the MSO circuit breaker monitoring solution delivered significant advantages across multiple operational dimensions:

 

• Hardware customization with software flexibility—Commercial-off-the-shelf (COTS) hardware customized through software down to the transistor level provided exceptional measurement quality without sacrificing reliability.
• ​Code portability—Existing CompactRIO code was seamlessly transitioned to the sbRIO platform, reducing development time and validation requirements.
• Optimized connectivity—The 320-pin connector of the sbRIO-9651 system-on-module (SOM) provided superior integration capabilities for long-term deployment.
• ​Scalable architecture—This “semi-custom” approach delivered an ideal balance of performance and cost-effectiveness for large-scale deployment across thousands of units.
• ​Comprehensive integration—NI software libraries enabled rapid I/O integration and communications protocol support, while RTOS capabilities ensured optimal, reliable performance.

 

​Ultimately, MSO’s enterprise software solution enabled the utility provider to meet the following goals:

 

• ​Constantly acquire and analyze key indicators in circuit breaker events
• ​Incorporate numerous, custom trigger definitions to enable detection and reporting of asset health and maintenance concerns continuously
• ​Design in safeguards, like short-term onboard storage to help protect against data loss when a system gets disconnected from the IIoT
• ​Deliver accurate and timely alerts through real-time analysis and reporting capabilities

 

​Improved Operational Efficiency

​Over the expected 20-year operating life of the CBA and CBOLM systems, the utility provider stands to save ~$400 million, with a nominal payback period of six years. A large portion of cost savings can be attributed to increased service reliability, enhancing grid stability and performance.

 

​The installation of CBOLM units across the utility service area is expected to achieve the following objectives:

 

• Extend the life of their existing circuit breaker population by one to three years
• ​Dramatically reduce customers outages
• ​Reduce the need for unplanned, reactive maintenance events

   

​Performing maintenance at the first sign of abnormal, or out-of-tolerance, circuit breaker behavior can help reduce further damage or deterioration. The CBOLM systems provide an unprecedented level of data insights into circuit breaker health, which enables the utility provider’s maintenance teams to prioritize the assets with the greatest need and intervene before a damaging or catastrophic failure can occur. These need-based insights and maintenance prioritization are expected to increase average circuit breaker life by one to three years.

 

​The availability of proactive, data-driven, condition-based service recommendations fundamentally reshaped service scheduling, enabling the utility provider to prioritize maintenance based on actual equipment condition rather than rigid, predetermined intervals. Reducing the frequency of unplanned maintenance activities has enabled this utility provider to reduce their cost of asset maintenance—and most importantly—limit the risks to their maintenance personnel by reducing the instances of unplanned maintenance activities.

 

​Since the CBOLM system captures and logs data any time a circuit breaker event occurs, it helps dramatically reduce the number of maintenance events that are needed to ensure the circuit breakers’ regulatory compliance. Maintenance teams have access to circuit breaker operating data via the centralized server, and therefore, can report on the regulatory compliance of breakers without needing to travel to the site.

 

​This enhanced approach to monitoring and managing assets will result in substantial, long-term cost savings for this utility provider: improved operational efficiency, enhanced regulatory compliance, and a designed-in approach for proactive maintenance with the promise of substantial long-term cost savings.

 

​Through transforming complex equipment monitoring needs into actionable intelligence, MSO and Cyth Systems empowered the utility provider to set new industry standards for predictive maintenance and grid management.