Optimal Structural Nested Models for Optimal Sequential Decisions

I describe two new methods for estimating the optimal treatment regime (equivalently, protocol, plan or strategy) from very high dimesional observational and experimental data: (i) g-estimation of an optimal double-regime structural nested mean model (drSNMM) and (ii) g-estimation of a standard single regime SNMM combined with sequential dynamicprogramming (DP) regression. These methods are compared to certain regression methods found in the sequential decision and reinforcement learning literatures and to the regret modelling methods of Murphy (2003). I consider both Bayesian and frequentist inference. In particular, I propose a novel “Bayes-frequentist compromise” that combines honest subjective nonor semiparametric Bayesian inference with good frequentist behavior, even in cases where the model is so large and the likelihood function so complex that standard (uncompromised) Bayes procedures have poor frequentist performance.