Unsupervised Reservoir Computing for Solving Ordinary Differential Equations
Abstract
There is a wave of interest in using unsupervised neural networks for solving differential equations. The existing methods are based on feedforward networks, {while} recurrent neural network differential equation solvers have not yet been reported. We introduce an unsupervised reservoir computing (RC), an echostate recurrent neural network capable of discovering approximate solutions that satisfy ordinary differential equations (ODEs). We suggest an approach to calculate time derivatives of recurrent neural network outputs without using backpropagation. The internal weights of an RC are fixed, while only a linear output layer is trained, yielding efficient training. However, RC performance strongly depends on finding the optimal hyperparameters, which is a computationally expensive process. We use Bayesian optimization to efficiently discover optimal sets in a highdimensional hyperparameter space and numerically show that one set is robust and can be used to solve an ODE for different initial conditions and time ranges. A closedform formula for the optimal output weights is derived to solve first order linear equations in a backpropagationfree learning process. We extend the RC approach by solving nonlinear system of ODEs using a hybrid optimization method consisting of gradient descent and Bayesian optimization. Evaluation of linear and nonlinear systems of equations demonstrates the efficiency of the RC ODE solver.
 Publication:

arXiv eprints
 Pub Date:
 August 2021
 arXiv:
 arXiv:2108.11417
 Bibcode:
 2021arXiv210811417M
 Keywords:

 Computer Science  Machine Learning;
 Computer Science  Neural and Evolutionary Computing;
 Physics  Computational Physics