We consider the problem of training logistic regression models for binary classification in information extraction and information retrieval tasks. Fitting probabilistic models for use with such tasks should take into account the demands of the taskspecific utility function, in this case the well-known F-measure, which combines recall and precision into a global measure of utility. We develop a...

We consider exact F tests for the hypothesis of null random factor effect in the presence of interaction under the two factor mixed models involved in the mixed models controversy. We show that under the constrained parameter (CP) model, even in unbalanced data situations, MSB/MSE (in the usual ANOV A notation) follows an exact F distribution when the null hypothesis holds. We also obtain an ex...

This paper evaluates the performance of different similarity measures in the context of document summarization. For this purpose in this paper a simple and effective sentence extractive technique is used. The proposed method is based on evaluation of relevance score of sentence. Many measures are available for the calculation of inter sentence relationships. To calculate a similarity between se...

Some new inequalities related to Jensen and Ostrowski inequalities for general Lebesgue integral are obtained. Applications for $f$-divergence measure are provided as well.

This paper describes a fuzzy union based approach for automatically evaluating machine generated extract summaries. The proposed method represents every sentence within a machine generated summary as a fuzzy set. Sentences in the reference summary are assigned membership grades in each of these fuzzy sets using cosine distance measure. Finally Fuzzy union (s-norm operation) is used to compute a...

We consider the problem of (macro) F-measure maximization in the context of extreme multi-label classification (XMLC), i.e., multi-label classification with extremely large label spaces. We investigate several approaches based on recent results on the maximization of complex performance measures in binary classification. According to these results, the F-measure can be maximized by properly thr...

Finding measures (or features) invariant to inevitable variations caused by non-linguistical factors (transformations) is a fundamental yet important problem in speech recognition. Recently, Minematsu [1, 2] proved that Bhattacharyya distance (BD) between two distributions is invariant to invertible transforms on feature space, and develop an invariant structural representation of speech based ...

The F-measure, which has originally been introduced in information retrieval, is nowadays routinely used as a performance metric for problems such as binary classification, multi-label classification, and structured output prediction. Optimizing this measure is a statistically and computationally challenging problem, since no closed-form solution exists. Adopting a decision-theoretic perspectiv...