Human assessment is often considered the gold standard in evaluation of translation systems. But in order for the evaluation to be meaningful, the rankings obtained from human assessment must be consistent and repeatable. Recent analysis by Bojar et al. (2011) raised several concerns about the rankings derived from human assessments of English-Czech translation systems in the 2010 Workshop on M...

In many languages, a voiceless obstruent cannot occur after a nasal. In some languages such a sequence is avoided through voicing assimilation, but in others through deletion of the nasal or of the obstruent, or changing the nasal into an oral stop. This variety of responses can be expressed in OT through different constraint rankings. Other ways of avoiding such a sequence, such as epenthesis ...

This paper presents the first rankings of nonacademic organizations according to their contributions to the INFORMS practice literature. Two rankings are given, each based on a different metric: visibility is the number of authors who list an organization as their primary affiliation; yield is the equivalent number of INFORMS practice papers attributable to each organization based on author pri...

In this paper, we show a new technique to analyze families of rankings. In particular, we focus on sports rankings and, more precisely, on soccer leagues. We consider that two teams compete when they change their relative positions in consecutive rankings. This allows to define a graph by linking teams that compete. We show how to use some structural properties of this competitivity graph to me...

This study takes a closer look at the Ranking Web of Universities and Ranking Web of Repositories rankings and tries to examine if there is a connection between these two rankings. Study is done by analyzing the success of University of Jyväskylä and the institutional repository JyX of the University in these rankings. Comparison shows that the JyX archive plays an important role in University’...

Most studies of Bayesian updating use experimental data. This paper uses a non-experimental data source– the voter ballots of the Associated Press (AP) college football poll, a weekly subjective ranking of the top 25 teams–to test Bayes’ rule as a descriptive model. I estimate the voters’ Bayesian posterior rankings for the first seven weeks of the 2006-08 seasons (a total of over 31,000 rankin...

Building on previous studies on perceptions of inequality, welfare and risk we investigate the structure of individuals' rankings of uncertain prospects in terms of risk and their relationship to individual preferences. We examine three interlinked propositions that are fundamental to the standard economic approach to risk: (i) that rankings by risk are simply the reverse of ranking by preferen...

ABSTRACT Rankings are ubiquitous in the online world today. As we have transitioned from finding books in libraries to ranking products, jobs, job applicants, opinions and potential romantic partners, there is a substantial precedent that ranking systems have a responsibility not only to their users but also to the items being ranked. To address these often conflicting responsibilities, we prop...

We introduce d0 as an optimization measure for ranking problems, such as information retrieval (IR). d0 is an intuitive, differentiable measure which is somewhat correlated with standard IR measures of performance. A set of experiments is described in which the weight used to combine the results of two IR systems is chosen by optimizing d0. This technique works about as well as optimizing “exac...

A ranking on a graph is an assignment of positive integers to its vertices such that any path between two vertices of the same rank contains a vertex of strictly larger rank. A ranking is locally minimal if reducing the rank of any single vertex produces a non ranking. A ranking is globally minimal if reducing the ranks of any set of vertices produces a non ranking. A ranking is greedy if, for ...