Period: 01/08/2024 – 30/11/2024

SECTION 1 – GENERAL TRENDS OF THE PROJECT

Brief summary of the project

With regard to the specific timeframe, it is below provided:

With reference to the work packages/tasks reported in Table 1, the following progress has been made during the considered timeframe:

• WP0 – Task 0.1. The final version of both the parameter varying linear model and the Simulink version of the plasma equilibrium code CREATE-NL have been made available to the control design tasks. The final versions of the two models have been made available on the TRAINER website https://trainer.dieti.unina.it/index.php, and constitute deliverable [D1] and [D2] of the projectWP1 – Task 1.1. The first version of DDPG agents for the basic magnetic control problems will be refined in the next months using the new plasma models developed in T0.1.
• WP1 – Task 1.3. First proposal of a data-driven shape controller based on a DDPG technique. In followingbimesters, the design of the controller will be refined by exploiting different reinforcement learning techniques, such as TRPO and SAC, and different tuning parameters, such as different reward functions.
• WP2 – Task 2.1. The final version of the two Neural Networks (NNs) developed to estimate the movement of the plasma along the unstable mode, one based on Multilayer Perceptrons (MLP) and the other one based on Extreme Learning Machines (ELM), have been delivered to estimate the movement of the plasma along the unstable mode.
  These NNs constitute deliverable [D3] of the project.
• WP2 – Task 2.2. The data-driven Vertical Stabilization (VS) system based on the Extremum Seeking (ES) algorithm, whose first version has been developed during bimesters 3 and 4, has been customized for the TCV tokamak, in order to make a proposal for experiments in 2025. The finalization of the architecture will be carried out after the experimental validation, since minor changes may be required to solve unforeseen issues that can occur during experiments.
• WP4 – Task 4.1. During bimesters 5 and 6:
  • the project PI has been invited to present the reference architecture for magnetic control at the 3^rd IAEA Technical Meeting on Plasma Disruptions and their Mitigation, that has been held at the ITER Headquarter (France) in September 2024 (https://conferences.iaea.org/event/380/timetable/#20240903.detailed). Other members of the Federico II Research Unit have presented results about the development of the plasma modeling tools at the same Technical Meeting.
  • In September 2024, several members of the Federico II Research Unit have presented contribution to the 33^rd Symposium on Fusion Technology (SOFT'24), held in Dublin (Ireland) - https://soft2024.eu/
  • A paper describing the architecture of the data-driven VS system has been published on International Journal Expert Systems with Applications, in August 2024.

Table 1 - TRAINER work packages and tasks.

[WP0] – Development of a fast control-oriented nonlinear plasma equilibrium code
[T0.1] – Development of a fast version of a plasma equilibrium code to be executed within the Simulink environment and of an approximate parameter varying linear model of the plasma/tokamak dynamics
[WP1] – Development of DRL control agents for basic magnetic control problems
[T1.1] – Development of a first version of DDPG agents for the basic magnetic control problems (i.e. Plasma Current Control and Vertical Stabilization) by exploiting both single linear models and parameter varying linear model of the plasma response  to model the environment
[T1.2] – Refinement of the agents developed by [T1.1], by exploiting the fast nonlinear equilibrium code
[T1.3] – Assessment of the possibility of developing a DDPG agent to solve problem the Plasma Shape Control problem
[WP2] – Development of data-driven plasma parameters estimators for adaptive model-free plasma vertical stabilization
[T2.1] – Development of the data-driven estimator of the plasma growth rate
[T2.2] – Development of an Extremum-seeking based VS system and assess the possibility to integrate it with the growth rate estimator developed by [T2.1]
[WP3] - Development of DRL-based tuning procedures to improve the robustness of model-based magnetic control algorithms
[T3.1] – Optimization of the controller gains for the plasma current, shape and VS system exploiting DDPG algorithms and fast nonlinear simulations with the nonlinear equilibrium code
[WP4] – Dissemination
[T4.1] – Dissemination

 

Names of the operational units involved in the implementation of the project 

• Research Unit (RU) University of Naples “Federico II” – led by the Principal Investigator, Prof. Gianmaria De Tommasi
• RU University of Naples “Parthenope” – led by Prof. Marco Ariola

 

 Description of the achievement of the objectives connected to the project and related outcomes

 The project activities of the considered timeframe were mainly related to:

• development and use of the plasma models for controller design (i.e., agent training) and validation;
• customization of the data-driven VS system based on Extremum Seeking for the TCV tokamak and preparation of an experiment proposal;
• Federico II RU purchased a desktop and started the process to purchase several laptops to support the project activities. This RU decided to withdraw the purchase of the servers originally planned, since the RU members can use servers that have been purchased using funds from another project.
• Federico II RU did the selection of two post-docs in July 2024 to work on the following topics included in the TRAINER project
  • Design and real-time implementation of the plasma magnetic control systems in tokamak devices (awarded to dr. Emilio Acampora);
  • Iterative learning control (ILC) for the design of tokamaka plasmas scenarios (awarded to dr. Domenico Frattolillo).

The selected post-docs have started their activity from November 2024.

• development of a data-driven shape controller based on the use of a DDPG agent;
• publication of the following papers/presentations at International conferences:
  • [P1] Dubbioso, A. Jalalvand, J. Wai, G. De Tommasi, E. Kolemen, "Model-free Stabilization via Extremum Seeking using a cost neural estimator," Expert Systems with Applications, vol. 258, pp. 125204, August 2024 (this paper describes the data-driven architecture of the VS system based on the Extremum Seeking algorithms that will be tested, if possible, on the TCV tokamak).
  • [P2] Mattei, R. Ambrosino, M. Ariola, G. De Tommasi, P. de Vries, A. Pironti, L. Zabeo, "Recent Developments in ITER Magnetic Control Algorithms," in 33rd Symposium on Fusion Technology (SOFT'24), Dublin, Ireland, September 2024. (about the reference architecture for plasma magnetic control).
  • [P3] Dubbioso, D. Ottaviano, F. Fiorenza, N. Ferron, G. Manduchi, R. Ambrosino, G. De Tommasi, "Rapid prototyping of control modules for the DTT Plasma Control System," in 33rd Symposium on Fusion Technology (SOFT'24), Dublin, Ireland, September 2024 (about the use of rapid prototyping procedure for the deployment of the plasma control systems components).
  • [P4] Fiorenza, F. Frattolillo, G. De Tommasi, C. Ingesson, S. Inoue, M. Mattei, Y. Miyata, A. Neto, A. Pironti, M. Takechi, H. Urano, "Validation of ITER magnetic diagnostic algorithms by using JT-60SA magnetic measurements," in 33rd Symposium on Fusion Technology (SOFT'24), Dublin, Ireland, September 2024 (about the validation of the plasma modeling tools on data from the Integrated Commissioning and Operation 1 of JT-60SA).
  • [P5] Dubbioso, G. De Tommasi, C. Galperti, F. Felici, S. Marchioni, A. Mele, A. Merle, A. Tenaglia, "Simulation validation of an Extremum Seeking-based Vertical Stabilization system for TCV," in 33rd Symposium on Fusion Technology (SOFT'24), Dublin, Ireland, September 2024 (about the customization of the data-driven VS system based on ES for the TCV tokamak).
  • [P6] E. di Grazia, M. Mattei, A. Pironti, “An Extended Kalman Filter for Plasma Equilibrium Reconstruction,” in 33rd Symposium on Fusion Technology (SOFT'24), Dublin, Ireland, September 2024 (about the exploitation of the plasma magnetic model to reconstruct plasma equilibrium).
  • [P7] Frattolillo, L. E. di Grazia, M. Mattei, F. Pesamosca , “Magnetic control strategies to reduce first wall heat loads in ITER,” in 33rd Symposium on Fusion Technology (SOFT'24), Dublin, Ireland, September 2024 (about the exploitation of the plasma magnetic model for the deployment of ITER-relevant advanced control strategies).
  • [P8] E. di Grazia, E. Fable, M. Mattei, F. Maviglia, M. Siccinio, H. Zohm, “Burn control strategies using plasma elongation control in DEMO,” in 33rd Symposium on Fusion Technology (SOFT'24), Dublin, Ireland, September 2024 (about the validation of the plasma model on DEMO scenarios).

 

Description of the carried out activities which are in compliance with the DNSH, Open Access principles as well as with gender, generational principles and with those of Equal opportunities;

In the considered timeframe, the journal paper [P1] has been published as Open Access. As for DNSH and gender, generational principles, and with those of Equal opportunities no specific compliance have been experienced, in this timeframe.

Description of the actions aimed at informing and disseminating knowledge

The main activities performed in the considered timeframe are:

1. participation pf the project PI to the 3^rd IAEA Technical Meeting on Plasma Disruptions as invited speaker to present the contribution titled “Control of elongated plasmas in superconducting tokamaks in the absence of in-vessel coils”;
2. participation to the SOFT 2024 conference to present the contributions [P2]-[P5];
3. participation to the ITER PCS design project meeting in November 2024, to present the main results achieved on plasma magnetic modeling and control
4. update of the contents of the website https://trainer.dieti.unina.it/.

 

SECTION 2 – PROGRESS OF ACTIVITIES

With regard to the specific timeframe, it is below provided:

Detailed description of activities carried out by each operational unit with a focus on the timeframe for their implementation

In the considered timeframe, the Federico II RU has carried out the following activities: 

1. delivery of (Task 0.1)
   1. the parameter varying linear model;
   2. the Simulink version of the plasma equilibrium code CREATE-NL;
2. delivery of the data-driven estimator of the plasma movement along the unstable
   mode (Task 2.1).

Concerning item i.a) a Linear Parameter Varying (LPV) plasma model has been delivered that allows to schedule different models as a function of the plasma current or time. In the latter case the model becomes time-varying (LTV). The model is provided as Simulink block (see Figure 1) that is setup running a Matlab script that initializes the required data structures. In order to setup the block a set of plasma linear models to be scheduled is needed. As an example, Figure 2 shows a comparison between the response of a LPV and a linear (LTI) model of the ITER tokamak when a 10 cm Vertical Displacement Event (VDE) is stabilized by a simply proportional-derivative VS system. It can be noticed that the responses are different, since the LPV model allows to better approximate the nonlinear behaviour of the plasma response.

Figure 1 - Simulink simulation scheme that include the LPV plasma model, that allows to schedule the response as function of the plasma current or of the time. This scheme includes a simple Proportional Derivative VS system to vertically stabilize the plasma and compare the response of the LPV model with those one of a linear model (see also Figure 2).

Figure 2 - Plasma vertical displacement for a 10 cm VDE when the LPV model of the ITER tokamak is used. A comparison with a linear model is shown.

Concerning item i.b), the nonlinear magnetic equilibrium code CREATE-NL has been delivered as a Simulink S-function. The final version allows relatively fast nonlinear simulations also on standard PCs. As an example, a simulation of the early ramp-up[1] phase of a 15 MA ITER discharge lasting 18 s (hence reaching I_p ~ 5 MA and including the limiter to divertor transition), at a sampling time of 2.5 ms, requires about 5 h on a standard PC. The test has been conducted on
the M3 iMac that has been purchased in August 2024 by the project. Figure 3 shows some of the output of a CREATE-NL nonlinear simulation for the ITER plasma.

It should be taken into account that the massive training of control agents should be run using
the LPV model (which executes faster than the nonlinear model).

On the other hand nonlinear plasma models should be used;

• either to refine the training (considering short time windows during plasma ramp-up and ramp-down or in case of fast disturbances acting during the flat-top);
• or to validate the agents by running high-fidelity simulations.

To this respect, the achieved performance are acceptable for training refinement and high-fidelity validations.

Figure 3 - Overview of a plasma simulation with the nonlinear equilibrium code CREATE-NL.

As far as the data-driven estimator of the plasma movement along the unstable mode (item ii), two NNs have been delivered, one based on Multilayer Perceptrons (MLP) and the other one based on Extreme Learning Machines (ELM). These NNs estimators have been trained on ITER synthetic data, generated using linear models. The NNs output, which is the plasma movement along the unstable mode, proved to enhance the performance of the ES-based VS system developed under Task 2.2 (see also [R1]). Figures 4 and 5 (which are taken from [R1]) show and example of VDE response obtained using the MLP and ELM estimator in the ES-based VS system.

 

 

Figure 4 - The response to a VDE of 6 cm applied to the model od the ITER tokamak when the MLP estimator replace the Kalman filter in the ES-based VS system.

Figure 5 - The response to a VDE of 6 cm applied to the model od the ITER tokamak when the ELM estimator replace the Kalman filter in the ES-based VS system.

In the same timeframe, Parthenope RU has worked on the development of a data-driven shape controller based on a DDPG agent. The control architecture reported in Figure 6 has been tested and validated for a DEMO case study with a plasma with diverted configuration. The proposed architecture is composed of both a model-based part and a data-driven one.

The model-based part of the shape controller has been developed by using a cascade approach. The outer loop computes the ex-vessel coil currents needed to control the plasma shape. The output of the shape controller feeds the ex-vessel coils currents controller. The inner loop allows us to decouple and control the dynamics of the currents of the ex-vessel coils. This control strategy allows us to use the voltages of the ex-vessel coils to track the currents needed to control the plasma shape. 

The data-driven part of the shape controller has been designed to give an additional term to the control action of the model-based controller to compensate for disturbances and uncertainties. The agent computes the control action from the actual value of the tracking error with respect to a reference plant model, which allows us to set the desired performance. The control agents of the two controllers are added to obtain the reference currents for the ex-vessel coils, as shown in the scheme of Figure 6.

 

Figure 6 - Block diagram of the proposed DDPG-based shape control system.

References

[R1] S. Dubbioso, A. Jalalvand, J. Wai, G. De Tommasi, E. Kolemen, "Model-free Stabilization via Extremum Seeking using a cost neural estimator," Expert Systems with Applications, vol. 258, pp. 125204, August 2024.

Description of potential changes to what has been originally approved mentioning the impacts on the aim of the intervention, on the achievement of intermediate and long- term goals, on the proposed actions for improvement

Other than the anticipation of Task 2.2 activity that has been already recorded in the previous report, and which does not affect the overall project objectives, no other changes to what has been originally approved were required.

Description of potential challenges encountered and of the proposed actions for improvement

With regard to the specific timeframe, no major challenges or issues have been encountered.

Brief description of potential publications

The following journal versions of the poster contributions to the SOFT 2024 Conference held in Dublin (Ireland) in September 2024 have been submitted in this timeframe or are still under preparation and will be submitted in December 2024:

• Dubbioso, G. De Tommasi, C. Galperti, F. Felici, S. Marchioni, A. Mele, A. Merle, A. Tenaglia, “Simulation validation of an Extremum Seeking-based Vertical Stabilization system for TCV.”
• Dubbioso, D. Ottaviano, F. Fiorenza, G. De Tommasi, N. Ferron, G. Manduchi, E. Acampora, R. Ambrosino, “Rapid prototyping of control modules for the DTT Plasma Control System.”
• Fiorenza, G. De Tommasi, D. Frattolillo, C. Ingesson, S. Inoue, S. Kojima, M. Mattei, A. Mele,
• Miyata, A. Neto, A. Pironti, O. Sauter, M. Takechi, H. Urano, “Preliminary validation of ITER magnetic diagnostic algorithms by using JT-60SA magnetic measurements.”
• Mattei, R. Ambrosino, M. Ariola, P. de Vries, G. De Tommasi, L. E. di Grazia, Y. Gribov,
  L. Pangione, A. Pironti, L. Zabeo, “Recent Developments in ITER Magnetic Control Algorithms.
• L.E. di Grazia, M. Mattei, A. Pironti, “An Extended Kalman Filter for Plasma Equilibrium Reconstruction.”
• Frattolillo, L. E. di Grazia, M. Mattei, F. Pesamosca , “Magnetic control strategies to reduce first wall heat loads in ITER.”
• E. di Grazia, E. Fable, M. Mattei, F. Maviglia, M. Siccinio, H. Zohm, “Burn control strategies using plasma elongation control in DEMO.”

SECTION 3 – COMMON INDICATORS

Below the updates on the indicator RRFCI 8 – “Number of researchers who work in research centres which are recipients of financial support (women; men; non-binary)” – as per the description in the guidelines included in the n. 34 MEF notification from the 17^th of October 2022.

Common indicators (RU University of Naples “Federico II”)	Planned value	Implemented value
Researchers who work in research centers which are recipients of financial support (women)	0,0	0,0
Researchers who work in research centers which are recipients of financial support (men)	1,1	0,63
Researchers who work in research centers which are recipients of financial support (non-binary)	0,0	0,0

 

Common indicators (RU University of Naples “Parthenope”)	Planned value	Implemented value
Researchers who work in research centers which are recipients of financial support (women)	0,0	0,0
Researchers who work in research centers which are recipients of financial support (men)	0,53	0,05
Researchers who work in research centers which are recipients of financial support (non-binary)	0,0	0,0

 

SECTION 4 – PREDICTIVE ANALYSIS AND FINAL COMMENTS

Below it is provided a description of the forecast scenario on the development of the project, any potential change which is deemed necessary for the future as well as comments on the document.

Predictive analysis

The plasma modelling tools have been delivered by Federico II RU, as expected (deliverables [D1] and [D2]). These models will be used to refine the DDPG agents for the basic magnetic control problems (see Task 1.1). The Federico II RU delivered also the data-driven estimator for the movement of the plasma along the vertical unstable mode (deliverable [D3]).

The data-driven VS system has been tailored for the TCV tokamak and a proposal for experiments has been submitted within the EUROfusion Tokamak Exploitation Work Package (WPTE): it is likely that first experimental validation will be carried out in the first half of 2025. Parthenope RU proposed the first version of a data-driven shape controller, which will be refined in the next months by exploiting different reinforcement learning techniques and different tuning parameters.

Final comments

The project is progressing as planned.

---

[1] The plasma current ramp-up and ramp-down phases represent the most challenging cases for plasma nonlinear simulation, if compared with plasma current flat-top phase. Indeed during plasma ramp-up and ramp-down the plasma configuration undergoes major changes, implying an increase of time needed to iteratively solve the Grad-Shafranov equation that returns the plasma equilibrium.