A Hybrid Dimension Reduction Based Linear Discriminant Analysis for Classification of High-Dimensional Data

Abstract

Linear discriminant analysis (LDA) is a notable classification algorithm thanks to its major success in many applications of the real-world. In spite of its successfulness for low-dimensional data, a dimension reduction is inevitable for its achievement with high-dimensional data, especially in which the number of features is more than the training sample size or close to the training sample size. Principal component analysis (PCA) plus LDA (PCA+LDA), a quite popular technique, is widely used for raising the classification performance of LDA over high-dimensional data. However, PCA ignores the label information in the data. On the other hand, the reduced dimensional data through PCA still includes indiscriminate (i.e., irrelevant) features. To cope with the dimensionality problem of LDA, a hybrid dimension reduction approach of supervised and unsupervised algorithms is proposed in this study. In the supervised part of the proposed hybrid dimension reduction method, called DBDERF+PCA, we propose to combine an ensemble classifier (i.e., random forest) with dichotomous binary differential evolution (DBDE), a recently proposed variant of binary differential evolution, by introducing a robust wrapper feature selection. On the other hand, unsupervised part of the proposed hybrid dimension reduction method utilizes PCA. The experimental results show that DBDERF+PCA outperforms PCA in terms of dimension reduction. Thereupon, this hybrid dimension reduction based LDA, called DBDERF+PCA+LDA, performs better than PCA+LDA and LDA in terms of run-time and classification performances.