How Enginuity Global Modernized Legacy Control Systems at QuarterNorth Energy’s Bullwinkle Production Platform for Operational Excellence and Safety in 45 Days
Rockwell Automation PlantPAx® Distributed Control System (DCS), including:
HMI/SCADA and Factory Talk® historian with SQL Server supporting 6,000 I/Os
Anti-virus and cluster management software
System integration by Enginuity Global, LLC
Benefits
Centralized monitor and control system resulting in higher yield, improved safety, and downtime-free operations for critical applications
Leveraged virtualization to consolidate 12 applications running on 6 servers onto a single, fault-tolerant Stratus ftServer compute platform
Contributed to a 22% reduction in unplanned downtime, increasing productivity
Reduced system response time by 1,000%, from more than 20 seconds to milliseconds, increasing control and safety
QuarterNorth Energy modernizes Bullwinkle platform for operational excellence
QuarterNorth Energy is a leading oil and gas producer that operates deep water assets in the Gulf of Mexico. Through innovation and a disciplined approach to oil and gas production management, the company specializes in operating diverse assets by using technology to gain efficiencies and generate profits. The Bullwinkle offshore production platform, currently the largest fixed offshore platform in the United States of America, is located close to Louisiana, and is strategic for tying new wells to existing export flow lines. As with many aged assets, the Bullwinkle platform had passed through several owners, ran on legacy infrastructure, and was using a working control system that had become outdated and difficult to support.
QuarterNorth engaged Enginuity Global, a system integrator with expertise in process automation, control, field services, and emergency support to maximize operations and minimize downtime, to modernize and centralize Bullwinkle’s infrastructure and control systems. With a two-phased approach, the Enginuity team migrated the platform from 9 legacy PLCs, 6 servers, and low bandwidth data communications, to a fully redundant platform-wide control anchored by one Stratus Edge Computing system to centralize operational applications and share data throughout the platform and to the enterprise. The team accomplished the significant transformation in 45 days from site acceptance to facility restart. The Stratus compute platform provides fault-tolerant computing to run the Bullwinkle’s critical applications with no downtime or data loss, is serviceable by Operational Technology (OT), and significantly lowered Total Cost of Ownership (TCO).
Upgrading an aging platform to an efficient asset
The existing Bullwinkle system presented several challenges to their operating model. Over years of continuous updates, the control system devolved into a decentralized architecture, which required more than 20 operators to roam the 384 foot tall platform to monitor screens spread out over several locations. Additionally, parts of the system were near obsolete, with aging components that would unexpectedly fail. Replacing parts was difficult—with the team at times resorting to sourcing older components on eBay—resulting in long lead times to find and replace parts. As a result, the system was continuously at risk of experiencing costly and unplanned downtime events.
To modernize and centralize the Bullwinkle systems used in control and safety operations, Enginuity Global worked with QuarterNorth to execute a two-phased project to replace the existing servers and workstations with a single redundant Stratus Edge Computing platform, consolidate disparate multi-brand software into a single Rockwell Automation PlantPAx automation and control suite, upgrade the existing Allen-Bradley® PLC-5® controllers to Rockwell Automation ControlLogix®, and update the existing daisy chained remote I/O connections to a dual, ring-based Ethernet network for full system redundancy throughout the platform.
According to Eric Belgard, Enginuity Lead for Automation and Controls, managing the process in phases “lessens the pain” of a major overhaul while also making it easier for companies to manage projects within their budgets.
The primary focus throughout this project was delivering a system that assured personnel safety, enhanced operator awareness, and increased reliability while ensuring supportability—essential for both the customer and the system integrators who support them. A key part of Enginuity’s approach included the Stratus ftServer, which provides the fault tolerance and scalability to meet QuarterNorth Energy’s current and future demands to run critical applications in a single compute platform. Not only does the Stratus platform deliver the performance required to run the multiple applications on Bullwinkle, but it also provides easier management for the platform’s operators, including Customer Replaceable Units (CRUs) that are easily installed by OT teams without incurring downtime. And, like QuarterNorth, Stratus support runs 24/7, enabling the quick resolution of support calls at any time. Once Enginuity evaluated the costs of several alternatives, they found the Stratus ftServer offered the greatest reliability and the lowest TCO.
“Having the Stratus server there gives us peace of mind because the Stratus server is truly fault tolerant. It’s one of the main reasons why we always choose to go Stratus server when available” Eric Belgard Lead Systems Engineer Enginuity Global, LLC
Virtualization and reliability with Stratus Edge Computing
The first phase in the modernization project focused on updating the automation and control software and replacing the multiple servers and workstations with a single Edge Computing platform. Prioritizing their focus on oil, well, and transport operations, QuarterNorth Energy wanted a solution that was simple to operate, had no failover time, and prevented data loss. The Stratus ftServer that Enginuity installed supports 16 thick and thin clients, replacing the six servers and 16 workstations that repeatedly failed and were at or past their end-of-life (EOL). A fault-tolerant compute platform, the ftServer provides hot-swappable components and automatic health monitoring. Using application virtualization, the team consolidated 12 applications onto the single Stratus system, further simplifying management and reducing costs. The longer lifespan of the ftServer—8 to 10 years—means they no longer have to invest in replacing obsolete hardware every 3 to 4 years.
Client/Workstation Overview
New process control system and safety shutdown system upgrade safety, compliance, and decision making
The second phase of the modernization project migrated the existing control system to connect to Rockwell Automation PlantPAx running on the Stratus ftServer. The aging Allen-Bradley PLC-5 units were outdated and had reached the limit of their capacity, with no room for additional logic. Control operations were combined with the safety monitoring system, creating challenges with compliance. And, response time in the decentralized system was slow. An operator would issue a command and then wait 10 to 20 seconds to see the results. In addition, there was no consistency in the multi-brand HMI interfaces, with each requiring different terminals or monitors and having different alarms and trends.
Enginuity replaced the existing PLC-5 units with a new redundant control system based on the PlantPAx DCS, separating the Process Control Systems (PCS) and Safety Shutdown System (SSDS) into separate processors and racks. They converted legacy PLC-5 controllers connected to the PlantPAx DCS using conversion kits from Rockwell Automation, allowing them to re-use existing wiring for more than 100 I/O panels and 6,000 I/O points. Enginuity then installed a new Ethernet network, organized into two redundant fiber optic rings. In addition to slashing the response time from tens of seconds to milliseconds—a 1000% improvement in speed—the new Ethernet network provides a modern, standards-based protocol that is not only faster and more reliable, but easier to support and maintain.
The modernized system dramatically improved the safety of the platform’s operations—a critical success factor for QuarterNorth Energy. Michael Kinzel, Automation and Controls Engineer, emphasized, “Reliable system response speed is critical for continuous operations.” As part of their safety management process, QuarterNorth performs simulations of different events and measures the time to respond and react. During a simulation of an abnormal process they noted that response time dropped from tens of seconds to less than 3 seconds. This remarkable reduction in response time means operators can react faster during unplanned events.
The new solution also enhances traceability, including the automatic tracking of alarms, bypasses, and other events—critical to ensuring safety and compliance. Because the previous system lacked this capability, operators had to manually build audit trails of who performed activities and when they occurred. The automatic tracking of all events in the new system, including who did what, when, and for how long, provides a better record of events, enhancing safety and supporting compliance. The resulting DCS is both performant and fully redundant, offering visibility and control, while making it easier and safer for operators to work on multiple systems from anywhere on the platform.
Redundant Ethernet I/O network
“The ftServer is a tank. It offers a simplistic approach to a complex system. Part of our motto at Enginuity is, ’We simplify a complex world.’ And so, Stratus servers definitely help reduce some of that complexity. And, we were able to save a substantial amount of money using them.” Eric Belgard Lead Systems Engineer Enginuity Global, LLC
Usability and serviceability built for OT
Another important advantage of the newly installed PlantPAx DCS system is the reduction in operator workload. Instead of having to learn and manage multiple interfaces, operators can now view data through standardized HMI screens. Improved data collection speeds up the data historization process, supporting faster and more informed decision-making. Leveraging virtualization to consolidate workloads from 12 different applications into one system eliminates islands of automation and supports application availability. Thick and thin clients managed with Rockwell Automation’s ThinManager® and located throughout the platform provide better access, allowing technicians to manage operations from any level or location on the platform. The new system also reduces the complexity of operations, both in the management and monitoring of systems as well as in their regular maintenance.
Due in part to this new architecture, the QuarterNorth Energy deepwater operations team has reduced the Bullwinkle production platform unplanned downtime by 22%, increasing productivity. Built-in redundancy—from the ftServer and the dual Ethernet networks to the Process Control System and Safety Shutdown System panels—ensure the system is running and available at all times. “We have noticed the number of calls reporting server or system downtime has dropped —especially in comparison to the amount of previous cases,” remarked Belgard.
Looking forward: A scalable solution with room for growth
Looking forward, Michael Kinzel sees several opportunities to further improve operations on the Bullwinkle platform. They plan to further streamline their infrastructure by porting applications running on several less reliable independent devices to the ftServer. Highlighting the Stratus system’s virtualization and system availability, Kinzel explained, “We can do this because we have confidence in the ftServer.” In fact, the solution works so well, QuarterNorth Energy is in the process of setting up a similar engagement with Enginuity to modernize another location. “This architecture really works,” summarized Kinzel.
Distributed Control System with ftServer Fault-Tolerant Platform
