An Orthogonal Least Squares (OLS) based feature selection method is proposed for both binomial and multinomial classification. The novel Squared Correlation Coefficient (SOCC) defined on Error Reduction Ratio (ERR) in OLS used as the ranking criterion. equivalence between canonical correlation coefficient, Fisher’s criterion, sum of SOCCs revealed, which unveils statistical implication ERR firs...