Effective methods of capacity control via uniform convergence bounds for function expansions have been largely limited to Support Vector machines, where good bounds are obtainable by the entropy number approach. We extend these methods to systems with expansions in terms of arbitrary (parametrized) basis functions and a wide range of regularization methods covering the whole range of general li...

A statistical model or a learning machine is called regular if the map taking a parameter to a probability distribution is one-to-one and if its Fisher information matrix is always positive definite. If otherwise, it is called singular. In regular statistical models, the Bayes free energy, which is defined by the minus logarithm of Bayes marginal likelihood, can be asymptotically approximated b...

The mean-variance theory of Markowitz (1952) indicates that large investment portfolios naturally provide better risk diversification than small ones. However, due to parameter estimation errors, one may find ambiguous results in practice. Hence, it is essential to identify relevant stocks to alleviate the impact of estimation error in portfolio selection. To this end, we propose a linkage cond...

The problem of model selection is considerably important for acquiring higher levels of generalization capability in supervised learning. In this article, we propose a new criterion for model selection, the subspace information criterion (SIC), which is a generalization of Mallows's C(L). It is assumed that the learning target function belongs to a specified functional Hilbert space and the gen...

The fuzzy structured element (FSE) theory is a very useful toolfor dealing with fuzzy multi-criteria decision making (MCDM)problems by transforming the criterion value vectors of eachalternative into the corresponding criterion function vectors. Inthis paper, some concepts related to function vectors are firstdefined, such as the inner product of two function vectors, thecosine of the included ...

A two component parametric mixture is proposed to model survival after an invasive treatment, when patients may experience different hazards regimes: a risk of early mortality directly related to the treatment and/or the treated condition, and a risk of late death influenced by several exogenous factors. The parametric mixture is based on Weibull distributions for both components. Different set...

In this paper we propose a novel general framework for unsupervised model adaptation. Our method is based on entropy which has been used previously as a regularizer in semi-supervised learning. This technique includes another term which measures the stability of posteriors w.r.t model parameters, in addition to conditional entropy. The idea is to use parameters which result in both low conditio...

Smooth boosting algorithms are variants of boosting methods which handle only smooth distributions on the data. They are proved to be noise-tolerant and can be used in the “boosting by filtering” scheme, which is suitable for learning over huge data. However, current smooth boosting algorithms have rooms for improvements: Among non-smooth boosting algorithms, real AdaBoost or InfoBoost, can per...

A new method for model selection for Gaussian Bayesian networks and Markov networks, with extensions towards ancestral graphs, is constructed to have good mean squared error properties. The method is based on the focused information criterion, and offers the possibility of fitting individualtailored models. The focus of the research, that is, the purpose of the model, directs the selection. It ...