Feature selection applicable to classification and clustering by the analysis of optical diffraction and entropy score discrimination

Abstract

Knowing the actual relevant features of a given data set not only can speed up the learning processes of classification or clustering algorithm, but also induce the higher prediction accuracy. Truly relevant selected features can make the prediction accuracy achieve 100%. However, it is not an easy task to distinguish the relevant features from the noisy features. This is because the selected relevant features must preserve the actual distribution and topological structure of the data space regardless of the original features. Therefore, a new feature selection based on unsupervised clustering and measure is proposed. The features are rearranged based on their relevant scores. This technique is called filter technique. Our approach is based on the observation that in any dimension, the distribution of clusters is similar to the scattering distribution of light passing through a set of vertical slits. The discrimination of data distribution is re-examined and evaluated using a simple observation motivated by the concept of optics diffraction. A property of the Fourier transform of probability density distribution is used. It is hypothesized that the features with high discrimination score are the relevant features. The criterion and algorithm are, then, extended to deal with data orientation whose direction of data alignment is defined by performing the discrimination evaluation on the bases locating towards the direction of data orientation. Then, the discrimination score of original features are computed. The key contributions from this research are: (1) new filter technique for unsupervised feature selection based on optical discrimination analysis, (2) new scoring of the filter technique for unsupervised feature selection, and (3) feasible capability to select features in both supervised classification and unsupervised clustering applications. Comparing with Laplacian score, SVD-Entropy, and LLDA-RFE, our experimental results show the efficacy of the proposed approach.