Infinite clean energy? DeepMind AI to control plasma in fusion reactor

DeepMind, the AI ​​subsidiary of Google parent Alphabet, has successfully reapplied its algorithms to a difficult scientific problem. In cooperation with the Swiss Plasma Center at the Swiss Federal Institute of Technology in Lausanne (EPFL), the UK-based company has now trained its deep reinforcement learning AI to control ultra-hot plasma in an experimental nuclear fusion reactor. .

one in the diary Nature According to published articles, the system can help physicists better understand how nuclear fusion works and potentially help accelerate the development of this endlessly dreamed-up source of clean energy. “This is one of the most challenging applications of reinforcement learning in a real system,” says DeepMind researcher Martin Riedmiller.

In nuclear fusion, the nuclei of hydrogen atoms are compressed to form heavier atoms such as helium, a process that also occurs in the Sun. An enormous amount of energy is generated in this process relative to a tiny amount. of starting material, which makes it a very efficient energy source in principle. Nuclear fusion is much cleaner and safer than fossil fuels or even conventional nuclear power. The latter uses nuclear fission, in which nuclei break apart, producing radioactive waste that must be disposed of.

Fourth state: plasma

However, controlling nuclear fusion in reactors is technically very difficult. The central problem: atomic nuclei repel each other. Bringing them together in a reactor is only possible at extremely high temperatures, often reaching hundreds of millions of degrees, hotter than at the center of the sun. At these temperatures, matter is neither solid nor liquid nor gas. Instead, it goes into a fourth state called plasma. It is an extremely hot particulate soup that simmers.

The task now is to hold the plasma together in a reactor long enough to extract energy from it. Inside the sun, the plasma is held together by gravity. On Earth, physicists and chemists have to use a variety of tricks. In a magnet-based reactor, the so-called tokamak, the plasma is trapped in a magnetic cage. Horizontal conductive loops enclose the doughnut-shaped reactor vessel, in the middle of which is an elongated solenoid magnet that induces a ring-shaped current in the reactor and thus further heats the plasma. via Checking the current in the coils. surrounding magnetic fields can form. This prevents the plasma from touching the walls of the reactor vessel, which could cool it down and damage the reactor at the same time.

But controlling the plasma requires constant monitoring and changing of the magnetic field, which previous controllers handle more or less well. Therefore, the DeepMind team trained their reinforcement learning algorithm on precisely this in a simulation. Reinforcement learning training should allow a software agent to behave in any situation in such a way as to pursue its goal as strictly as possible, without explicitly writing all possible actions for all possible situations. This basically works by trial and error: you try an action, if it works, you’re more likely to select it next time. If it is not effective, its chance of selection is reduced. After a sufficient number of attempts, the agent learns an optimal “policy”.

Virtual Reactor Exercise

After the AI ​​learned to properly control and change the shape of the plasma using magnetic impulses in the virtual reactor, an experiment began. In a small experimental reactor, EPFL researchers let the algorithm take over, the Variable Configuration Tokamak (TCV) located at the university. The result: the AI ​​could be seen to be perfectly capable of controlling the real reactor without further adjustments by the humans. However, the attempt was very short due to the system: the AI ​​controlled the plasma for a total of two seconds; that’s how long the TCV can run before it overheats.

For the control to work, the DeepMind AI neural network records 90 different measured values ​​about 10,000 times per second. These show, among other things, the current shape and position of the plasma. The voltage of the 19 magnets of the TCV is set accordingly. The resulting feedback loop works much faster than the previously used PID controller. But, above all, it is much more flexible, because it allows to generate and control a wide variety of plasma configurations with a single control structure.

To speed up the system, the AI ​​was split into two neural networks. In the simulation, a large network first learned how to control the reactor. These capabilities were then transferred to a smaller, faster network that controls the reactor itself (the so-called “critical/actor” model). However, because the simulation used to train the software becomes too imprecise over certain parameter ranges, the control software excludes such “uncertain areas” as a precaution.

AI to better experiment with plasma controls

However, the success is impressive. “It’s an incredibly powerful technique,” says Jonathan Citrin of the Netherlands Institute for Fundamental Energy Research, who is familiar with the study. “It’s an important first step in a very exciting direction.” The research team believes that using AI to control plasma could also make it easier to experiment with different conditions in such reactors. This, in turn, helps to better understand the process and potentially accelerate the development of commercial nuclear fusion reactors.

The investigation is progressing, but only slowly. Recently, researchers using the JET experimental fusion reactor showed that they can generate a self-sustaining fusion reaction with deuterium and tritium. However, to create a fusion reaction in a tokamak that produces more energy than was put into it, you need a large reactor like ITER with superconducting magnets. However, it won’t be ready until 2035 at the earliest. It remains to be seen whether private companies, which are also working on smaller fusion reactors and in some cases pursuing completely different physical concepts, are actually faster. So far, the race is still completely open.

However, the joy of experimenting could increase. “With this type of control system, we can take a risk that would otherwise be problematic,” says Ambrogio Fasoli, director of the Swiss Plasma Center and president of the Eurofusion consortium. Human operators are often unwilling to force plasma beyond certain limits. “There are events that we absolutely must avoid because they damage hardware.” However, if you are sure you have a control system that doesn’t overload the technology despite being on the edge, you might as well explore entirely new possibilities. “We could speed up the investigation.”

(jle)