Distributed Algorithms for LinearlySolvable Optimal Control in Networked MultiAgent Systems
Abstract
Distributed algorithms for both discretetime and continuoustime linearly solvable optimal control (LSOC) problems of networked multiagent systems (MASs) are investigated in this paper. A distributed framework is proposed to partition the optimal control problem of a networked MAS into several local optimal control problems in factorial subsystems, such that each (central) agent behaves optimally to minimize the joint cost function of a subsystem that comprises a central agent and its neighboring agents, and the local control actions (policies) only rely on the knowledge of local observations. Under this framework, we not only preserve the correlations between neighboring agents, but moderate the communication and computational complexities by decentralizing the sampling and computational processes over the network. For discretetime systems modeled by Markov decision processes, the joint Bellman equation of each subsystem is transformed into a system of linear equations and solved using parallel programming. For continuoustime systems modeled by Itô diffusion processes, the joint optimality equation of each subsystem is converted into a linear partial differential equation, whose solution is approximated by a path integral formulation and a sampleefficient relative entropy policy search algorithm, respectively. The learned control policies are generalized to solve the unlearned tasks by resorting to the compositionality principle, and illustrative examples of cooperative UAV teams are provided to verify the effectiveness and advantages of these algorithms.
 Publication:

arXiv eprints
 Pub Date:
 February 2021
 DOI:
 10.48550/arXiv.2102.09104
 arXiv:
 arXiv:2102.09104
 Bibcode:
 2021arXiv210209104W
 Keywords:

 Computer Science  Machine Learning;
 Computer Science  Multiagent Systems;
 Computer Science  Robotics;
 Electrical Engineering and Systems Science  Systems and Control;
 Mathematics  Optimization and Control