Learning Conditional Independence Relations from a Probabilistic Model

We consider the problem of learning conditional independencies, expressed as a Markov network, from a probabilistic model. An eecient algorithm employing a greedy search has been developed earlier with promising empirical results. However, two issues were not addressed. First, the reason why the myopic search works so well globally has not been fully understood. Second, whether the algorithm can nd a correct Markov network in all cases has not been formally established. In this paper, we prove that, for any given probabilistic model, the algorithm will always produce a Markov network whose structure is an independence map of the underlying model and whose associated probability distribution is identical to the underlying model. The proof also ooers deeper insight into the algorithm's working mechanism. As the problem of learning a minimal independence map of a given probabilistic model is NP-hard in general, our polynomial time algorithm does not guarantee minimality in all cases. We show that, however , if the given probabilistic model belongs to a subclass that has a singly connected independence map, the algorithm will always produce a Markov network whose structure is a minimal independence map.