Deep Quaternion Networks
Abstract
The field of deep learning has seen significant advancement in recent years. However, much of the existing work has been focused on real-valued numbers. Recent work has shown that a deep learning system using the complex numbers can be deeper for a fixed parameter budget compared to its real-valued counterpart. In this work, we explore the benefits of generalizing one step further into the hyper-complex numbers, quaternions specifically, and provide the architecture components needed to build deep quaternion networks. We develop the theoretical basis by reviewing quaternion convolutions, developing a novel quaternion weight initialization scheme, and developing novel algorithms for quaternion batch-normalization. These pieces are tested in a classification model by end-to-end training on the CIFAR-10 and CIFAR-100 data sets and a segmentation model by end-to-end training on the KITTI Road Segmentation data set. These quaternion networks show improved convergence compared to real-valued and complex-valued networks, especially on the segmentation task, while having fewer parameters
- Publication:
-
arXiv e-prints
- Pub Date:
- December 2017
- DOI:
- 10.48550/arXiv.1712.04604
- arXiv:
- arXiv:1712.04604
- Bibcode:
- 2017arXiv171204604G
- Keywords:
-
- Computer Science - Neural and Evolutionary Computing;
- Computer Science - Computer Vision and Pattern Recognition
- E-Print:
- IJCNN 2018, 8 pages, 1 figure