Period: 01/12/2024 – 31/03/2025

SECTION 1 – GENERAL TRENDS OF THE PROJECT

Brief summary of the project

With regard to the specific timeframe, it is below provided:

With reference to the work packages/tasks reported in Plasma Physics and Controlled Fusion, in March 2025.

•  An invited session on Plasma modeling and control in magnetically confined fusion devices has been submitted and accepted to the Joint 9th IFAC Symposium on System Structure and Control, 19th IFAC Workshop on Time Delay Systems and 2nd IFAC Workshop on Control of Complex System (https://sssc-tds-cosy-2025.sciencesconf.org/). The invited session has been proposed by the members of the TRAINER project: Sara Dubbioso, Federico Fiorenza and Domenico Frattolillo.
• A paper describing a magnetic control system composed of both a model-based part and a data-driven one has been prepared and submitted for publication on the International Journal of Dynamics and Control, in December 2024.

Table 1, the following progress has been made during the considered timeframe:

• WP1 – Task 1.1. Refining of DDPG agents for the basic magnetic control problems, i.e. vertical stabilization and plasma current control, by using the new plasma models developed in T0.1.
• WP1 – Task 1.3. Refining of the shape controller by using the TRPO reinforcement learning technique, and different tuning parameters, such as different reward functions.
• WP2 – Task 2.2. Further assessment of the possible validation of the data-driven Vertical Stabilization (VS) system based on the Extremum Seeking (ES) algorithm on the TCV tokamak. Impact of control and communication delays has been evaluated by means of an Hardware-in-the-loop (HIL) setup and by testing it in the TCV simulation & validation environment.
• WP4 – Task 4.1. During bimesters 7 and 8:
  • the journal versions of the contributions presented to the 33^rd Symposium on Fusion Technology (SOFT'24) in September 2024 (https://soft2024.eu/) have been prepared and submitted for publication to Fusion Engineering and Design. In particular the following paper are currently under review:
    • Mattei, R. Ambrosino, M. Ariola, G. De Tommasi, P. de Vries, A. Pironti, L. Zabeo, "Recent Developments in ITER Magnetic Control Algorithms.”
    • Dubbioso, D. Ottaviano, F. Fiorenza, N. Ferron, G. Manduchi, R. Ambrosino, G. De Tommasi, "Rapid prototyping of control modules for the DTT Plasma Control System."
    • Fiorenza, F. Frattolillo, G. De Tommasi, C. Ingesson, S. Inoue, M. Mattei, Y. Miyata, A. Neto, A. Pironti, M. Takechi, H. Urano, "Validation of ITER magnetic diagnostic algorithms by using JT-60SA magnetic measurements."
    • Dubbioso, G. De Tommasi, C. Galperti, F. Felici, S. Marchioni, A. Mele, A. Merle, A. Tenaglia, "Simulation validation of an Extremum Seeking-based Vertical Stabilization system for TCV."
    • E. di Grazia, M. Mattei, A. Pironti, “An Extended Kalman Filter for Plasma Equilibrium Reconstruction.”
    • Frattolillo, L. E. di Grazia, M. Mattei, F. Pesamosca , “Magnetic control strategies to reduce first wall heat loads in ITER.”
    • E. di Grazia, E. Fable, M. Mattei, F. Maviglia, M. Siccinio, H. Zohm, “Burn control strategies using plasma elongation control in DEMO.”
  • A paper describing the experimental results obtained at TCV with a current limit avoidance system designed exploiting the model developed by Task 0.1 has been prepared and submitted for publication on the International Journal Plasma Physics and Controlled Fusion, in March 2025.
  • An invited session on Plasma modeling and control in magnetically confined fusion devices has been submitted and accepted to the Joint 9th IFAC Symposium on System Structure and Control, 19th IFAC Workshop on Time Delay Systems and 2nd IFAC Workshop on Control of Complex System (https://sssc-tds-cosy-2025.sciencesconf.org/). The invited session has been proposed by the members of the TRAINER project: Sara Dubbioso, Federico Fiorenza and Domenico Frattolillo.
  • A paper describing a magnetic control system composed of both a model-based part and a data-driven one has been prepared and submitted for publication on the International Journal of Dynamics and Control, in December 2024.

 

Table 1 - TRAINER work packages and tasks.

[WP0] – Development of a fast control-oriented nonlinear plasma equilibrium code
[T0.1] – Development of a fast version of a plasma equilibrium code to be executed within the Simulink environment and of an approximate parameter varying linear model of the plasma/tokamak dynamics
[WP1] – Development of DRL control agents for basic magnetic control problems
[T1.1] – Development of a first version of DDPG agents for the basic magnetic control problems (i.e. Plasma Current Control and Vertical Stabilization) by exploiting both single linear models and parameter varying linear model of the plasma response  to model the environment
[T1.2] – Refinement of the agents developed by [T1.1], by exploiting the fast nonlinear equilibrium code
[T1.3] – Assessment of the possibility of developing a DDPG agent to solve problem the Plasma Shape Control problem
[WP2] – Development of data-driven plasma parameters estimators for adaptive model-free plasma vertical stabilization
[T2.1] – Development of the data-driven estimator of the plasma growth rate
[T2.2] – Development of an Extremum-seeking based VS system and assess the possibility to integrate it with the growth rate estimator developed by [T2.1]
[WP3] - Development of DRL-based tuning procedures to improve the robustness of model-based magnetic control algorithms
[T3.1] – Optimization of the controller gains for the plasma current, shape and VS system exploiting DDPG algorithms and fast nonlinear simulations with the nonlinear equilibrium code
[WP4] – Dissemination
[T4.1] – Dissemination

 

Names of the operational units involved in the implementation of the project 

• Research Unit (RU) University of Naples “Federico II” – led by the Principal Investigator, Prof. Gianmaria De Tommasi
• RU University of Naples “Parthenope” – led by Prof. Marco Ariola

 

 Description of the achievement of the objectives connected to the project and related outcomes

 The project activities of the considered timeframe were mainly related to:

• development and use of the plasma models for controller design (i.e., agent training) and validation;
• customization of the data-driven VS system based on Extremum Seeking for the TCV tokamak and preparation of an experiment proposal;
• Federico II RU purchased a desktop and started the process to purchase several laptops to support the project activities. This RU decided to withdraw the purchase of the servers originally planned, since the RU members can use servers that have been purchased using funds from another project.
• Federico II RU did the selection of two post-docs in July 2024 to work on the following topics included in the TRAINER project
  • Design and real-time implementation of the plasma magnetic control systems in tokamak devices (awarded to dr. Emilio Acampora);
  • Iterative learning control (ILC) for the design of tokamaka plasmas scenarios (awarded to dr. Domenico Frattolillo).

The selected post-docs have started their activity from November 2024.

• further validation of the data-driven VS system based on Extremum Seeking for the TCV tokamak by means of HIL simulations and by testing it withing the TCV simulation & validation environment;
• further validation of the control-oriented models developed by Task 0.1 on the TCV tokamak. The developed models have been used to design a current-limit avoidance systems based on online Quadratic Programming (QP) optimization.
• Federico II RU purchased several laptops to support the work of the some of the team members new comers.
• Refinement of the data-driven controller based on the use of control-oriented models developed by Task 0.1 and TRPO agent;
• publication of the following papers:
  • [P1] Dubbioso, G. De Tommasi, N. Ferron, C. Galperti, G. Manduchi, S. Marchioni, A. Mele, A. Pironti, “Hardware-in-the-loop validation of an Extremum Seeking-based system for vertical stabilization of tokamak plasmas,” accepted for presentation at Joint 9th IFAC Symposium on System Structure and Control, 19th IFAC Workshop on Time Delay Systems and 2nd IFAC Workshop on Control of Complex System, Gif-sur-Yvette, France, July 2025 (about the HIL validation of the ES-based VS for TCV).
  • [P2] Neri, P. Zumbolo, R. Albanese, "Accurate plasma boundary calculation using linear triangular finite elements," Computational Physics, vol. 311, pp. 109574, June 2025. (about the development of plasma model for magnetic diagnostics)
  • [P3] E. di Grazia, C. Vincent, M. Mattei, F. Felici, L. Kogan, A. Mele, "Iterative learning optimization and control of MAST-U breakdown and early ramp-up scenarios," Optimization and Engineering, 2025. (about the experiment validation of the control-oriented model on MAST-U).
  • [P4] E. di Grazia, E. Fable, M. Mattei, M. Siccinio, F. Maviglia, H. Zohm, "Burn control strategies using plasma elongation in DEMO," Fusion Engineering and Design, vol. 215, pp. 114976, June 2025. (about the validation of the control-oriented model on DEMO).

 

Description of the carried out activities which are in compliance with the DNSH, Open Access principles as well as with gender, generational principles and with those of Equal opportunities;

No specific compliance with the DNSH, Open Access principles as well as with gender, generational principles, and with those of Equal opportunities has been experienced in the considered timeframe.

Description of the actions aimed at informing and disseminating knowledge

The main activities performed in the considered timeframe are:

1. preparation of a proposal for an invited session on Plasma modeling and control in magnetically confined fusion devices at the Joint 9th IFAC Symposium on System Structure and Control, 19th IFAC Workshop on Time Delay Systems and 2nd IFAC Workshop on Control of Complex System (https://sssc-tds-cosy-2025.sciencesconf.org/) that will be held at Gif-sur-Yvette (France) in July 2025. The invited session has been accepted.
2. update of the contents of the website https://trainer.dieti.unina.it/.

 

SECTION 2 – PROGRESS OF ACTIVITIES

With regard to the specific timeframe, it is below provided:

Detailed description of activities carried out by each operational unit with a focus on the timeframe for their implementation

In the considered timeframe, the Federico II RU has carried out the following activities: 

1. delivery of (Task 0.1)
   1. the parameter varying linear model;
   2. the Simulink version of the plasma equilibrium code CREATE-NL;
2. delivery of the data-driven estimator of the plasma movement along the unstable
   mode (Task 2.1).

In the considered timeframe, the Federico II RU has further validated the ES-based VS system for TCV by means of Hardware-in-the-loop (HIL) validations aimed at estimating the maximum control delay that can be tolerated in real-time, i.e. the robustness against control delays. Figure 1 shows the adopted HIL setup. The considered real-time environment is based on MARTe2 [R1] and MDSplus [R2], which represent a popular solution in the fusion community for the deployment of real-time control and data acquisition systems. By exploiting the Automatic Code Generation capability of the adopted real-time framework, HIL simulations are performed by exporting the ES-based VS as Generic Application Module (GAM) running on the real-time MARTe2 host running Linux running a vanilla Ubuntu version 22.04.4 LTS, without any real-time patch and/or task segregation policy. Furthermore, a discrete-time version of the TCV plant model is executed in real-time by the dSpace SCALEXIO high-performance target, permitting to test also the communication interface. In the considered setup, communication between the control host and the SCALEXIO target is implemented through a non-isolated UDP communication channel.

Figure 1 - Considered setup for rapid prototyping and validation via HIL simulations

Preliminary simulations in the Simulink environment have been performed to estimate the maximum control delay D_max for the two different choices of the ES control parameters, namely Set A and Set B. Simulink simulations performed with the linear model of pulse #81277 at a fixed sampling time equal to T_s = 100 μs (corresponding to a sampling frequency fs = 10 kHz), return the following bounds for the maximum delay for the two considered settings

400 μs < D^A_max < 500 μs ,
200 μs < D^B_max < 300 μs .

Such estimations have been confirmed by the HIL simulations. Indeed, the use of a non-real-time operating system allowed to account for random control delays during the HIL simulations, which have been in turn used to confirm the estimation of D_max when network communication delays are also included. In fact, due to the absence of any real-time provision, spurious events cause anomalous jitters during the execution of the control thread, as shown in Figure 3 and Figure 6, where the thread execution times recorded by MARTe2 are shown for the HIL simulations run with the two considered ES settings.

In particular, when the ES parameters of Set A are used, thread execution delays up to 250 μs summed to the network communication delay do not cause any problem to the VS, being the communication delay negligible with respect to jitter. On the other hand, the anomalous spike that occurs at t = 0.34 s, and corresponds to a delay of about 600 μs, causes the loss of plasma vertical stability. In fact, when such an anomalous spike occurs, the behavior of the in-silico plant running on the SCALEXIO target becomes unstable, as shown in Figure 5 Similarly, when the ES parameters of Set B are considered, the anomalous execution delay of about 850 μs that occurs around 14.16 s, causes the loss of the vertical control while delays up to about 100 μs do not cause any problem to the ES-based control, as reported in Figure 4.

Figure 2 - Thread execution time (measured in μs) of the VS controller recorded by the MARTe2 application during the HIL simulations, when the ES parameters of Set A are used. Before the anomalous spike that corresponds to a control delay of about 600 μs, and that causes the lost of vertical stability, the average execution time is 100 μs with a jitter < 50 μs. The spurious spikes occur since the MARTe2 application runs on a non real-time Linux without any process isolation technique.

Figure 3 - Thread execution time (measured in μs) of the VS controller recorded by the MARTe2 application during the HIL simulations, when the ES parameters of Set B are used. In this case the anomalous corresponds to a control delay of about 850 μs, which also causes the lost of vertical stability

Figure 4 - Time traces recorded with the SCALEXIO real-time target, for the HIL of the ES-based VS when the parameters from Set A are used. The voltage and the current in the control coils (G coils) are reported, as well as the variations (with respect to the equilibrium values) of both the vertical position of the plasma centroid and of the plasma current. Vertical stability is lost when an anomalous delay of about 600 μs is introduced in the control loop by the non-real-time Linux OS used as MARTe2 host

Figure 5 - Time traces recorded with the SCALEXIO real-time target, for the HIL of the ES-based VS when the parameters from Set B are used. In this case the vertical stability around 14.16 s.

 

In the same timeframe, Parthenope RU has worked on the refinement of the data-driven shape controller by using a TRPO agent. The control architecture shown in Figure 6 has been tested and validated for a DEMO case study with a plasma with diverted configuration. The proposed architecture is composed of both a model-based part and a data-driven one.

During the training phase, the agent has been trained by using a reward function based on the processing of the shape tracking error with a smooth max function and a sigmoid-like nonlinear function. First of all, the current value of the observation vector is processed by the following smooth max function

then the value of the reward is obtained by considering the following sigmoid-like nonlinear function

Where b, c₁, c₂ m_sh > 0 are tuning parameters, ρ(t_k) is a boolean variable, i.e.ρ(t_k) = 1 if

Through the boolean variable ρ(t_k), we supply a penalty m_sh  if the tracking errors do not have acceptable values, and we prematurely stop the episode, since we consider the task failed. Note that the smooth max function (12) represents a smooth, differentiable approximation of the maximum function for large positive values of the parameter b. The function (13), ifρ(t_k) = 1, returns rewards in the range 0 < r(t_k) < 1 →  when f(t_k)→ 0. Indeed, during the training the agent learns the control policy to minimize the maximum value of the tracking error of the controlled gaps. 

 

Figure 6 - Block diagram of the proposed TRPO-based shape control system

References

[[R1] A. C. Neto et al., "MARTe: A multiplatform real-time framework," IEEE Transactions on Nuclear Science, 57(2), 479–486, 2010.
[R2] G. Manduchi et al., "Integration of Simulink, MARTe and MDSplus for rapid development of real-time applications," Fusion Engineering and Design, 96, 645–648, 2015

Description of potential changes to what has been originally approved mentioning the impacts on the aim of the intervention, on the achievement of intermediate and long- term goals, on the proposed actions for improvement

Other than the anticipation of Task 2.2 activity that has been already recorded in the Q2 report, and that did not affect the overall project objectives, no other changes to what has been originally approved were required.

Description of potential challenges encountered and of the proposed actions for improvement

With regard to the specific timeframe, no major challenges or issues have been encountered.

Brief description of potential publications

In this timeframe and, the following journal paper has been prepared and submitted for publication to Plasma Physics and Controlled Fusion:

Frattolillo, A. Mele, C. Galperti, L. E. di Grazia, M. Mattei, S. Coda, G. De Tommasi, A. Pironti, A. Tenaglia, P. de Vries; L. Pangione, L. Zabeo, "Implementation of an ITER-relevant QP-based Current Limit Avoidance algorithm in the TCV tokamak. (About the use of the developed control-oriented plasma model for the design of a model-based current limit avoidance system for TCV).

SECTION 3 – COMMON INDICATORS

Below the updates on the indicator RRFCI 8 – “Number of researchers who work in research centres which are recipients of financial support (women; men; non-binary)” – as per the description in the guidelines included in the n. 34 MEF notification from the 17^th of October 2022.

Common indicators (RU University of Naples “Federico II”)	Planned value	Implemented value
Researchers who work in research centers which are recipients of financial support (women)	0,0	0,0
Researchers who work in research centers which are recipients of financial support (men)	1,2	0,89
Researchers who work in research centers which are recipients of financial support (non-binary)	0,0	0,0

 

Common indicators (RU University of Naples “Parthenope”)	Planned value	Implemented value
Researchers who work in research centers which are recipients of financial support (women)	0,0	0,0
Researchers who work in research centers which are recipients of financial support (men)	0,53	0,18
Researchers who work in research centers which are recipients of financial support (non-binary)	0,0	0,0

 

SECTION 4 – PREDICTIVE ANALYSIS AND FINAL COMMENTS

Below it is provided a description of the forecast scenario on the development of the project, any potential change which is deemed necessary for the future as well as comments on the document.

Predictive analysis

The data-driven VS system customized for the TCV tokamak has been further tested and validated in simulation, also by means of HIL simulations. It is still expected to carry out the first experimental validation in the first half of 2025. Parthenope RU refined the proposed data-driven shape controllers, by exploiting different reinforcement learning techniques and different tuning parameters. In the next months a further evaluation of the performance will be performed.

Final comments

The project is progressing as planned.