Some Results on Penalized Spline Estimation in Generalized Additive and Semiparametric Models

In spline regression models, the smoothness of the tted model depends on the knots via their location and how many knots there are. An alternative to knot selection is to keep all the knots, but to restrict their in uence on the tted model, that is, to constrain the values of these coe cients which correspond to the spline basis functions. For example, we can bound the squared L2 norm by some constant. In this case, for a classical additive regression model with d covariates, the tted values are given by b f =G Y where G =X(X X +A ) X T , A = blockdiag1 j d( jDj), Dj = diag(0pj 1;1Kj), X is the full design matrix of the covariates (up to polynomial degree pj) and the spline basis functions f(xji jk)+g , where j1; : : : ; jKj is a set of knots for the jth covariate. Since the amount of smoothing is determined by , this is the vector of smoothing parameters. See also Marx and Eilers (1998). Some other penalty functions are proposed by Ruppert and Carroll (1997). We obtain closed form approximations to the bias and variance of the estimators, not just for additive models, but for generalized additive models (GAM). The results can for example be used to provide rough starting values for the smoothing parameters and they provide some backup for commonly used degrees of freedom approximations.