Pitch-Informed Instrument Assignment Using a Deep Convolutional Network with Multiple Kernel Shapes
Abstract
This paper proposes a deep convolutional neural network for performing note-level instrument assignment. Given a polyphonic multi-instrumental music signal along with its ground truth or predicted notes, the objective is to assign an instrumental source for each note. This problem is addressed as a pitch-informed classification task where each note is analysed individually. We also propose to utilise several kernel shapes in the convolutional layers in order to facilitate learning of efficient timbre-discriminative feature maps. Experiments on the MusicNet dataset using 7 instrument classes show that our approach is able to achieve an average F-score of 0.904 when the original multi-pitch annotations are used as the pitch information for the system, and that it also excels if the note information is provided using third-party multi-pitch estimation algorithms. We also include ablation studies investigating the effects of the use of multiple kernel shapes and comparing different input representations for the audio and the note-related information.
- Publication:
-
arXiv e-prints
- Pub Date:
- July 2021
- DOI:
- 10.48550/arXiv.2107.13617
- arXiv:
- arXiv:2107.13617
- Bibcode:
- 2021arXiv210713617L
- Keywords:
-
- Computer Science - Sound;
- Computer Science - Information Retrieval;
- Computer Science - Machine Learning;
- Computer Science - Neural and Evolutionary Computing;
- Electrical Engineering and Systems Science - Audio and Speech Processing
- E-Print:
- 4 figures, 4 tables and 7 pages. Accepted for publication at ISMIR Conference 2021