Physics-Guided Actor-Critic Reinforcement Learning for Swimming in Turbulence
Abstract
Turbulent diffusion causes particles placed in proximity to separate. We investigate the required swimming efforts to maintain a particle close to its passively advected counterpart. We explore optimally balancing these efforts with the intended goal by developing and comparing a novel Physics-Informed Reinforcement Learning (PIRL) strategy with prescribed control (PC) and standard physics-agnostic Reinforcement Learning strategies. Our PIRL scheme, coined the Actor-Physicist, is an adaptation of the Actor-Critic algorithm in which the Neural Network parameterized Critic is replaced with an analytically derived physical heuristic function (the physicist). This strategy is then compared with an analytically computed optimal PC policy derived from a stochastic optimal control formulation and standard physics-agnostic Actor-Critic type algorithms.
- Publication:
-
arXiv e-prints
- Pub Date:
- June 2024
- DOI:
- 10.48550/arXiv.2406.10242
- arXiv:
- arXiv:2406.10242
- Bibcode:
- 2024arXiv240610242K
- Keywords:
-
- Electrical Engineering and Systems Science - Systems and Control;
- Computer Science - Machine Learning;
- Nonlinear Sciences - Chaotic Dynamics;
- Physics - Fluid Dynamics;
- Statistics - Machine Learning
- E-Print:
- 23 pages, 6 figures