A Bayesian View of Challenges in Feature Selection: Feature Aggregation, Multiple Targets, Redundancy and Interaction

Earlier, we formulated a Bayesian approach to Feature Subset Selection using Bayesian networks, which jointly estimate the posteriors of Markov Blanket Memberships (MBMs), Markov Blanket Sets (MBSs), and Markov Blanket Subgraphs (MBGs) for a given target variable. These results of the Bayesian Multilevel Analysis of relevance (BMLA) correspond respectively to a model-based pairwise relevance, relevance of sets, and to the interaction models of relevant variables. In this paper we discuss applications of the Bayesian approach to new challenges in relevance analysis. First, we formulate refined levels in BMLA by introducing the concepts of k-MBSs and k-MBGs, which are intermediate, scalable model properties expressing relevance. Second, we consider the extension of BMLA to multiple targets. Third, we introduce and investigate a score for feature redundancy and interaction based on the decomposability of the structure posterior. Finally, we overview the problems of conditional and contextual relevance. We demonstrate the concepts and methods in the field of the genomics of asthma.