In many machine learning task, we have data Z from some distribution p and the task is to minimize the risk: R(f) = EZ∼p[`(f(Z), Z)] (11.1) where ` is a loss function of interest, e.g. in classification Z = (X,Y ) and we use 0/1 loss `(f(Z), Z) = 1f(X)6=Y , in regression Z = (X,Y ) and we use squared error `(f(Z), Z) = (f(X) − Y ) and in density estimation Z = X and we use negative log likeliho...

We show several high-probability concentration bounds for learning unigrams language model. One interesting quantity is the probability of all words appearing exactly k times in a sample of size m. A standard estimator for this quantity is the Good-Turing estimator. The existing analysis on its error shows a high-probability bound of approximately O ( k √ m ) . We improve its dependency on k to...

let be a random sample from a normal distribution with unknown mean and known variance the usual estimator of the mean, i.e., sample mean is the maximum likelihood estimator which under squared error loss function is minimax and admissible estimator. in many practical situations, is known in advance to lie in an interval, say for some in this case, the maximum likelihood estimator changes and d...

in a subclass of the scale-parameter exponential family, we consider the sequential pointestimation of a function of the scale parameter under the loss function given as the sum of the weightedsquared error loss and a linear cost. for a fully sequential sampling scheme, second order expansions areobtained for the expected sample size as well as for the regret of the procedure. the former resear...

When training the parameters for a natural language system, one would prefer to minimize 1-best loss (error) on an evaluation set. Since the error surface for many natural language problems is piecewise constant and riddled with local minima, many systems instead optimize log-likelihood, which is conveniently differentiable and convex. We propose training instead to minimize the expected loss, ...

The standard loss functions used in the literature on probabilistic prediction are the log loss function and the Brier loss function; however, any proper loss function can be used for comparison of prediction algorithms. This note shows that the log loss function is most selective in that any prediction algorithm that is optimal for a given data sequence (in the sense of the algorithmic theory ...

in this paper, a bayesian approach is proposed for shift point detection in an inverse gaussian distribution. in this study, the mean parameter of inverse gaussian distribution is assumed to be constant and shift points in shape parameter is considered. first the posterior distribution of shape parameter is obtained. then the bayes estimators are derived under a class of priors and using variou...

Deep learning has become the method of choice in many application domains of machine learning in recent years, especially for multi-class classification tasks. The most common loss function used in this context is the cross-entropy loss, which reduces to the log loss in the typical case when there is a single correct response label. While this loss is insensitive to the identity of the assigned...

Let be a random sample from a normal distribution with unknown mean and known variance The usual estimator of the mean, i.e., sample mean is the maximum likelihood estimator which under squared error loss function is minimax and admissible estimator. In many practical situations, is known in advance to lie in an interval, say for some In this case, the maximum likelihood estimator...