Debugging using Orthogonal Gradient Descent
Abstract
In this report we consider the following problem: Given a trained model that is partially faulty, can we correct its behaviour without having to train the model from scratch? In other words, can we ``debug" neural networks similar to how we address bugs in our mathematical models and standard computer code. We base our approach on the hypothesis that debugging can be treated as a twotask continual learning problem. In particular, we employ a modified version of a continual learning algorithm called Orthogonal Gradient Descent (OGD) to demonstrate, via two simple experiments on the MNIST dataset, that we can infact \textit{unlearn} the undesirable behaviour while retaining the general performance of the model, and we can additionally \textit{relearn} the appropriate behaviour, both without having to train the model from scratch.
 Publication:

arXiv eprints
 Pub Date:
 June 2022
 DOI:
 10.48550/arXiv.2206.08489
 arXiv:
 arXiv:2206.08489
 Bibcode:
 2022arXiv220608489C
 Keywords:

 Computer Science  Machine Learning;
 Computer Science  Artificial Intelligence