Supervised feature extraction algorithm based on improved polynomial entropy

Based on information entropy theory, a novel feature extraction algorithm based on improved polynomial entropy (IPE) is set up. Firstly, the concepts and their properties of information entropy and cross entropy (CE) are analysed and studied. On this foundation, we prove that symmetrical cross entropy (SCE) proposed here based on CE satisfies three axioms of the distance, i.e. nonnegativity, symmetry and triangle inequation. So SCE is a kind of distance measure, which can be used to measure the degree of variation between two random variables. Secondly, for convenience, we propose a new concept of improved polynomial entropy (IPE) based on polynomial entropy (PE), and explain that IPE is equivalent to SCE. Thirdly, we make IPE separability criterion of the classes for feature extraction, and design a novel feature extraction algorithm based on IPE. Fianlly, the experimental results demonstrate that the algorithm proposed here is valid and reliable, and it provides a new research approach for feature extraction, data mining and pattern recognition.