City Research Online

Abstract

Deep reinforcement learning (DRL) applied to thermal convection control consistently produces degenerate actuation: wall-temperature policies whose outputs are saturated, pseudo-random, or spatially incoherent. Two compounding deficiencies are responsible: multilayer-perceptron policies that discard spatial flow structure, and memoryless policies that cannot distinguish self-induced flow changes from background evolution. Together they degrade the actuation into forms whose relation to the convective topology cannot be read off and which are not realisable at the actuator, even when cell coalescence (the merging of convection rolls into fewer, larger structures), which would reduce Nu, is accessible to boundary actuation.
The present framework addresses both causes through four targeted design choices: convolutional policy networks, Gated Recurrent Unit (GRU) memory, off-policy training (TD3 / MADDPG), and action-smoothness constraints. A systematic 2 × 2 factorial design isolates the contribution of each component. On Rayleigh–Bénard convection at Ra=10,000, all four configurations achieve cell coalescence and reduce Nu to as low as 1.83 (26% below the uncontrolled baseline) in 350 episodes, without the full-field data augmentation required by prior work. Crucially, coalescence is achieved even by the single-agent configuration, demonstrating that the multi-agent formulation is not a prerequisite once the policy architecture is sufficiently expressive. Applied to double-diffusive convection in the salt-finger regime, the framework spontaneously discovers a travelling-wave actuation whose phase speed adapts to the evolving mixing state of the flow, enhancing heat transfer by 19.1% and reducing salinity variance by 21.0%.

Publication Type:	Article
Additional Information:	© 2026 The Authors. Published by Elsevier Inc. This is an open access article distributed under the terms of the Creative Commons CC-BY license, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Publisher Keywords:	Active heat and flow control; Deep reinforcement learning; Multi-agent control; Partial observability; Smooth actuation; Rayleigh–Bénard convection; Double-diffusive convection; Cell coalescence
Subjects:	T Technology > TA Engineering (General). Civil engineering (General)
Departments:	School of Science & Technology School of Science & Technology > Department of Engineering
SWORD Depositor:	Symplectic Administrator

Export

Downloads