Assessing evaluation procedures for individual researchers: The case of the Italian National Scientific Qualification

The National Scientific Qualification (ASN) has been introduced as a prerequisite for applying for a tenured associate or full professor position at state-recognized Italian universities. The ASN is meant to attest that an individual has reached a suitable level of scientific maturity to apply for professorship positions. A controversial aspect of the ASN is its reliance on quantitative indicators as one of the parameters that must be used to assess applicants. In a previous works we have undertaken a comprehensive analysis of the publicly available results, focusing on the quantitative indicators of applicants. Here we complete the analysis by focusing on the content of the application forms and of the final reports that contain a written assessment of each applicant by a five-member panel. The application forms provide useful information about the age distribution of applicants, and the distribution of publication types and other qualifications for each scientific discipline. Furthermore, analysis of co-qualifications (i.e., the same individual getting multiple qualification) unveils the relationships and partial overlaps that exist between disciplines. The content of final reports are evaluated through simple metrics to understand whether they provide useful feedback to the applicants. In the final part of this work we contrast the methodology used in the National Scientific Qualification (ASN) with the best practices that are recommended for assessment of individual researchers.