CODAC
The "invisible system" that makes all things possible
10 Feb 2020
-
Pat Brans
It is easy to spot all the big equipment going into ITER; what is not so visible is the underlying software that makes the equipment come alive.
The Controls Division has to ensure that every local control system works with central conventional control, CODAC. In this photo, the team is checking the protection logic for the high voltage relays during the commissioning of ITER's electrical distribution system.
Local control software manages each of the big ITER systems—everything from electricity and cooling water to all the critical systems inside the Tokamak Complex. It also manages some of the smaller systems—right down to the one that regulates the flow of potable water in buildings.
CODAC is based on the open source system EPICS, which is the standard in the field of control systems engineering. The local control systems also follow this protocol, and are designed and developed to conform to a number of standards, including the interface definition for how the systems integrate with CODAC. But within those boundaries there is a lot of flexibility in how the different suppliers design and implement the control software. "The control system requirements are generic enough to be open to some interpretation," says Wallander.
"Each of these local control systems has to interface with central conventional control, CODAC (Control, Data Access and Communication)," says Anders Wallander, Head of the Controls Division. "We have 170 local control systems delivered by 101 suppliers —plus the CODAC itself, which makes a total of 171 control systems. It is a major undertaking to get all this software to work together."
How control systems fit into the commissioning process
The Controls Division is responsible for designing and delivering the CODAC system and for making sure all the local control systems—many of which are delivered with a major system as an in-kind contribution—interface correctly with CODAC. Integrating the software takes place during the commissioning process, when equipment and software are handed over from suppliers and Domestic Agencies to the ITER Organization.
CODAC is based on the open source system EPICS, which is the standard in the field of control systems engineering. The local control systems also follow this protocol, and are designed and developed to conform to a number of standards, including the interface definition for how the systems integrate with CODAC. But within those boundaries there is a lot of flexibility in how the different suppliers design and implement the control software. "The control system requirements are generic enough to be open to some interpretation," says Wallander.
"Commissioning starts when you switch things on and nothing works exactly as planned," says Anders Wallander, head of the Controls Division. This image captures the moment when the team first attempted to operate high voltage circuit breakers from the CODAC human machine interface.
The Controls Division inherits the local systems once they are handed off. They have to get it to work with the CODAC, and they have to maintain it. This means analyzing the source code, and running tests at the various test facilities set up specifically for that purpose. Based on what they see in the source code and what they find in the tests, the Controls Division makes a decision on whether or not to accept the software during commissioning.
"The best case handoff process is when we are in close communication with whoever developed the software," says Wallander. "When it doesn't work right away, we have to ask the software engineers a lot of questions. This is where personal relationships really make a difference."
First three systems commissioned
Before the first local system commissioning could begin, infrastructure had to be in place. "We depend on building structures and network infrastructure to connect the 170 systems that are spread around the site," says Wallander. "We use a dual star architecture, with a central point—the control room—that links to all the other places, very roughly in the shape of a star. We call it 'dual' because the network is redundant to provide fault tolerance."
However, the main control room will not be in place until 2022. To overcome this challenge, a temporary control room had to be put into place. (The temporary control room is really a "virtual control room" in the sense that it is several different rooms connected together.)
Connecting the 400 kV local control system to CODAC, and beginning to check signals (sensors and actuators).
Once the infrastructure was in place, the electricity was switched on and the first commissioning project—electrical distribution, along with its local control system—could begin. This was in June 2018.
"Commissioning starts when you switch things on and nothing works exactly as planned," says Wallander. "We all had to put in a lot of effort during the second half of 2018. But we were successful after six months."
Since January 2019, CODAC has been running 24/7, interfacing with the local control system to manage the electrical system. "It often comes down to engineers and other people who just want to get the systems to work together," says Wallander. "You put all these people in a building. They don't know each other, but they have a common goal to work towards. Very quickly, and very naturally, a team spirit develops. It was a long effort, with a lot of challenges. But that team spirit was a major factor in our success."
CODAC has now reached several major milestones, including the integration of 3 out of 170 local control systems. The next project, underway now, is the commissioning of the cooling water system.