Mutual Information Feature Extractors for Neural Classiiers
نویسنده
چکیده
This paper presents and evaluates two linear feature extractors based on mutual information. These feature extractors consider general dependencies between features and class labels, as opposed to statistical techniques such as PCA which does not consider class labels and LDA, which uses only simple rst order dependencies. As evidenced by several simulations on high dimensional data sets, the proposed techniques provide superior feature extraction and better dimensionality reduction while having similar computational requirements.
منابع مشابه
A Neural Network Based Hybrid System for Detection, Characterization and Classi cation of Short-Duration Oceanic Signals
Automated identiication and classiication of short-duration oceanic signals obtained from passive sonar is a complex problem because of the large variability in both temporal and spectral characteristics even in signals obtained from the same source. This paper presents the design and evaluation of a comprehensive classiier system for such signals. We rst highlight the importance of selecting a...
متن کاملLinear feature extractors based on mutual information
This paper presents and evaluates two linear feature extractors based on mutual information. These feature extractors consider general dependencies between features and class labels, as opposed to well known linear methods such as PCA which does not consider class labels and LDA, which uses only simple low order dependencies. As evidenced by several simulations on high dimensional data sets, th...
متن کاملProbabilistic Knowledge Transfer for Deep Representation Learning
Knowledge Transfer (KT) techniques tackle the problem of transferring the knowledge from a large and complex neural network into a smaller and faster one. However, existing KT methods are tailored towards classification tasks and they cannot be used efficiently for other representation learning tasks. In this paper a novel knowledge transfer technique, that is capable of training a student mode...
متن کاملLocal discriminant bases
We describe an extension to the \best-basis" method to construct an orthonormal basis which maximizes a class separability for signal classiication problems. This algorithm reduces the dimen-sionality of these problems by using basis functions which are well localized in time-frequency plane as feature extractors. We tested our method using two synthetic datasets: extracted features (expansion ...
متن کاملVector Quantizer Signal Transform
This paper deals with the problem of combination of Neural Networks (NN) and traditional statistical pattern classiiers. It is shown that a Neural Network can be used to replace the vector quantizer (VQ) and some feature extraction and feature reduction modules in a discrete pattern recognition system. A criterion for training the NN-weights and the classiier jointly is derived, leading to the ...
متن کامل