Generalized multiple classifier systems with local discriminant bases

Abstract

In recent years, the most successful and pervasive technique for pattern recognition and machine learning is a special family of classifiers known as multiple classifier systems. They have all demonstrated consistent - in some cases, remarkable improvements in predictive accuracy over single classifier systems. This dissertation extends the traditional concept in many aspects: improved recognition accuracy of multiple classifier systems through the use of linear transforms; robust generalized channel coding for multiple classifier systems; and optimal combining of predictions. The first contribution of this dissertation is an exploration of the use of frames, which are overcomplete sets of vectors, to form efficient discriminant representations, called Local Discriminant Frame Expansion (LDFE). The scheme is a simple and efficient method suitable for extending the Local Discriminant Bases (LDB) feature extraction algorithm into Multiple Description Coding (MDC) framework. To combat misclassification, preassigned amounts of redundancy are added to the original data during the feature extraction process. Unequal discriminant assignment is implemented by varying the amount of redundancy with the importance of data. For the second contribution, this dissertation develops three extensions of the original ECOC method based on more generalized concatenated coding schemes. These are the attempts to improve classification through the concatenations of two or more heterogeneous multiple classifier systems. Other contribution is the optimization of combining of the predictions. In particular, a new weighted combining scheme is investigated. The algorithm utilizes a ridge estimator with statistically tunning parameter. Finally, a face recognition task is performed by using LDB with neural networks. Based on the key observation on the high interpolation power of neural networks, a collection of transform networks is constructed, in which it can be interpreted as both the frameworks of incremental learning (at the output level) and Bayesian model averaging.