SARSA(0) Reinforcement Learning over Fully Homomorphic Encryption
Abstract
We consider a cloud-based control architecture in which the local plants outsource the control synthesis task to the cloud. In particular, we consider a cloud-based reinforcement learning (RL), where updating the value function is outsourced to the cloud. To achieve confidentiality, we implement computations over Fully Homomorphic Encryption (FHE). We use a CKKS encryption scheme and a modified SARSA(0) reinforcement learning to incorporate the encryption-induced delays. We then give a convergence result for the delayed updated rule of SARSA(0) with a blocking mechanism. We finally present a numerical demonstration via implementing on a classical pole-balancing problem.
- Publication:
-
arXiv e-prints
- Pub Date:
- February 2020
- DOI:
- 10.48550/arXiv.2002.00506
- arXiv:
- arXiv:2002.00506
- Bibcode:
- 2020arXiv200200506S
- Keywords:
-
- Electrical Engineering and Systems Science - Systems and Control;
- Computer Science - Cryptography and Security
- E-Print:
- 7 pages, 2 figures, submitted to SICE ISCS 2021