Comment on ‘‘Inverse groundwater modeling for hydraulic conductivity estimation using Bayesian model averaging

[1] Tsai and Li [2008] assert that the Bayesian information criterion (BIC) [Schwarz, 1978] is better suited for comparing models having different parameters than is the Kashyap criterion (KIC) [Kashyap, 1982] because a Fisher information term in the latter may rank models with relatively large parameter estimation uncertainties higher than other models. We start by noting that KIC reduces asymptotically to BIC as the number of observations becomes large relative to the number of adjustable model parameters [Ye et al., 2008]. If Tsai and Li [2008] were correct in their assertion, this would imply that it is better to treat a finite set of data as if it were theoretically infinite, a proposition that is logically unappealing and not necessary in practice. [2] The Fisher information term imbues KIC with desirable model selection properties not shared by BIC [Ye et al., 2008]: it sometimes prefers more complex models than does BIC because of its unique ability to discriminate between models not only on the basis of their goodness of fit to observational data and number of parameters but also on the quality of the available data and of the parameter estimates. To appreciate this role of the Fisher information term, it must not be considered in isolation as do Tsai and Li [2008] but rather in the context of all terms entering into KIC as do Ye et al. [2008]. The purpose of this comment is to elaborate on the discussion of Ye et al. [2008] by explaining further why the tendency of KIC to prefer models with relatively large parameter estimation uncertainty is a strength rather than a weakness. [3] In a manner analogous to that of Sivia and Skilling [2006], we present a simple example which helps elucidate the role played by the Fisher information term in KIC and allows us to offer general observations regarding more complex applications, such as the groundwater inverse modeling analysis of Tsai and Li [2008]. Consider two models, A and B, having one adjustable parameter each, m and l, respectively. Bayes’ theorem implies that the ratio between the posterior model probabilities, conditioned on an observation vector D, is