Smoothing Spline Semi-parametric Nonlinear Regression Models

We consider the problem of modeling the mean function in regression. Often there is enough knowledge to model some components of the mean function parametrically. But for other vague and/or nuisance components, it is often desirable to leave them unspecified and to be modeled nonparametrically. In this article, we propose a general class of smoothing spline semi-parametric nonlinear regression models (SNRM) which assumes that the mean function depends on parameters and nonparametric functions through a known nonlinear functional. SNRMs are natural extensions of both parametric and nonparametric regression models. They include many popular nonparametric and semi-parametric models such as the partial spline, varying coefficients, projection pursuit, single index, multiple index and shape invariant models as special cases. Building on reproducing kernel Hilbert spaces (RKHS), the SNRMs allow us to deal with many different situations in a unified fashion. We develop a unified estimation procedure based on minimizing penalized likelihood using Gauss-Newton and backfitting algorithms. Smoothing parameters are estimated using the generalized cross-validation (GCV) and generalized maximum likelihood (GML) methods. We derive Bayesian confidence intervals for the unknown functions. A generic and user-friendly R function is developed to implement our estimation and inference procedures. We illustrate our methods ∗Yuedong Wang (email: [email protected]) is Professor, Department of Statistics and Applied Probability, University of California, Santa Barbara, California 93106. Chunlei Ke (email: [email protected]) is Statistician, Amgen Inc, One Amgen Center Drive, Thousand Oaks, CA 91320. Yuedong Wang’s research was supported by a grant from the National Science Foundation (DMS-0706886). Address for correspondence: Yuedong Wang, Department of Statistics and Applied Probability, University of California, Santa Barbara, California 93106. We thank Dr. Genton for sending us the R Hydrae data. We thank the editor, the associate editor and a referee for constructive comments that substantially improved an earlier draft. 1 with analyses of three real data sets and evaluate finite-sample performance by simulations.