Interpretable Rule Discovery Through Bilevel Optimization of SplitRules of Nonlinear Decision Trees for Classification Problems
Abstract
For supervised classification problems involving design, control, other practical purposes, users are not only interested in finding a highly accurate classifier, but they also demand that the obtained classifier be easily interpretable. While the definition of interpretability of a classifier can vary from case to case, here, by a humanly interpretable classifier we restrict it to be expressed in simplistic mathematical terms. As a novel approach, we represent a classifier as an assembly of simple mathematical rules using a nonlinear decision tree (NLDT). Each conditional (nonterminal) node of the tree represents a nonlinear mathematical rule (splitrule) involving features in order to partition the dataset in the given conditional node into two nonoverlapping subsets. This partitioning is intended to minimize the impurity of the resulting child nodes. By restricting the structure of splitrule at each conditional node and depth of the decision tree, the interpretability of the classifier is assured. The nonlinear splitrule at a given conditional node is obtained using an evolutionary bilevel optimization algorithm, in which while the upperlevel focuses on arriving at an interpretable structure of the splitrule, the lowerlevel achieves the most appropriate weights (coefficients) of individual constituents of the rule to minimize the net impurity of two resulting child nodes. The performance of the proposed algorithm is demonstrated on a number of controlled test problems, existing benchmark problems, and industrial problems. Results on two to 500feature problems are encouraging and open up further scopes of applying the proposed approach to more challenging and complex classification tasks.
 Publication:

arXiv eprints
 Pub Date:
 August 2020
 arXiv:
 arXiv:2008.00410
 Bibcode:
 2020arXiv200800410D
 Keywords:

 Computer Science  Machine Learning;
 Computer Science  Neural and Evolutionary Computing;
 Statistics  Machine Learning
 EPrint:
 Total 26 pages and 30 figures. Main Paper: 12 pages, 12 figures. Supplementary Document: 14 pages, 18 figures