Isotonic Recursive Partitioning

Isotonic regression is a well-known nonparametric tool for fitting monotonic models and has been studied from both theoretical and practical aspects for several decades, with applications in psychology, medicine, biology, among others. However, it has enjoyed only limited interest in recent years in the context of modern statistical applications. We believe the two major reasons for this limited attention are computational difficulties on large data and statistical difficulties (overfitting). We present a novel algorithmic approach to isotonic regression that addresses these concerns in a manner that is both practically useful and of independent methodological and algorithmic interest. Our new algorithm for isotonic regression is based on recursively partitioning the predictor space through solution of progressively smaller best cut subproblems. This creates a regularized sequence of isotonic models of increasing model complexity that converges to the global isotonic regression solution. The models along the sequence are often more accurate than the unregularized isotonic regression model because of the complexity control they offer. We offer quantification of this complexity control through estimation of degrees of freedom along the path. We also develop efficient methods for generating the global solution through this sequence of structured subproblems, as each subproblem is equivalent to a network flow problem for which efficient algorithms exist. The success of the regularized models in prediction and our algorithms favorable computational properties are demonstrated through a series of simulated and real data experiments.