Qualitative Scientometrics

When scientometricians are asked about the relationship between citations and quality, they often argue along the lines that citations could be seen as indicators of use and that that this implies usefulness and impact on other research, which in turn is an argument for using them as indicators of quality. This paper questions the implicit linearity of such a ‘one-dimensional model’ of representing quality by quantity from a number of standpoints. First, the use of citations as well as any indicator that is used is performative in the sense that those getting measured by them, i.e. researchers or university administrators, will adapt their behavior to perform well on the scales that are used, either for recognition or for monetary reward. Second, equating citation counts with quality might imply a notion that they scale together such that low citation rates equals low quality and that a high rate implies high quality. Third, and most important, drawing from theories in STS it could be argued that scientometric indicators and measures of scientific achievement are co-produced, in the sense that development of performance based or results based indicators and what is to be regarded as high quality will be found to increasingly coincide with each other as a consequence of the performative idiom mentioned above. It is suggested that more work on the qualitative aspects of scientometric data and is called for.