We study the rates of convergence from empirical surrogate risk minimizers to the Bayes optimal classifier. Specifically, we introduce the notion of consistency intensity to characterize a surrogate loss function and exploit this notion to obtain the rate of convergence from an empirical surrogate risk minimizer to the Bayes optimal classifier, enabling fair comparisons of the excess risks of d...

We consider the problem of estimating the probability of error in multi-hypothesis testing when MAP criterion is used. This probability, which is also known as the Bayes risk is an important measure in many communication and information theory problems. In general, the exact Bayes risk can be difficult to obtain. Many upper and lower bounds are known in literature. One such upper bound is the e...

In recent years, we have studied information properties of various types of mixtures of probability distributions and introduced a new type, which includes previously known mixtures as special cases. These studies are disseminated in different fields: reliability engineering, econometrics, operations research, probability, the information theory, and data mining. This paper presents a holistic ...

The risk, or probability of error, of the classifier produced by the AdaBoost algorithm is investigated. In particular, we consider the stopping strategy to be used in AdaBoost to achieve universal consistency. We show that provided AdaBoost is stopped after n iterations—for sample size n and ν < 1—the sequence of risks of the classifiers it produces approaches the Bayes risk if Bayes risk L∗ > 0.

We design a new learning algorithm for the Set Covering Machine from a PAC-Bayes perspective and propose a PAC-Bayes risk bound which is minimized for classifiers achieving a non trivial margin-sparsity trade-off.

Consider the problem of estimating normal means from independent observations with known variances, possibly different from each other. Suppose that a second-level normal model is specified on the unknown means, with the prior means depending on a vector of covariates and the prior variances constant. For this two-level normal model, existing empirical Bayes methods are constructed from the Bay...

An empirical Bayes pooling method is used to combine and compare estimates of the value of a statistical life (VSL). The data come from 40 selected studies published between 1974 and 2002, containing 197 VSL estimates. The estimated composite distribution of empirical Bayes adjusted VSL has a mean of $5.4 million and a standard deviation of $2.4 million. The empirical Bayes method greatly reduc...

We introduce a strategy for quantifying and synthesising uncertainty about elements of a risk assessment using Bayes linear methods. We view the population of subjective belief structures and the use of Bayes linear adjustments as a flexible and transparent tool for risk assessors who want to quantify their uncertainty about hazard based on disparate sources of information. For motivation, we u...