Negative Results in Computer Science Evaluations

In Computer Science, properties of formal theories that model real-world phenomena can be formally demonstrated using logic formal systems, e.g., given a proof of the best case complexity of a problem, or a demonstration of the soundness and completeness of a solution. Additionally, as in other Natural Sciences, characteristics of a theory can be empirically evaluated following the scientific method which provides procedures to systematically conduct experiments and to test hypotheses about these characteristics. Formally proven properties or empirically confirmed hypotheses can be accepted as accounting of known facts, while falsifiable statements that cannot be validated correspond to negative and inconclusive results. In this talk, we first discuss the different types of negative results that can be obtained during the formal and empirical validation of Computer Science approaches, e.g., contra-examples of theorems, intractability and undecidability of a problem, or statistically non significant results. Next, we analyze the reasons that may conduct to observe negative results, and more importantly, the relevance of publishing negative results is discussed. Moreover, we attempt to aware our attendees about the tendency of camouflaging negative results as positive results by non-evaluating problematic solutions or redefining a problem. Finally, we encourage the definition of guidelines for reporting results that more that being seen as negative, should be considered as new challenges that will allow for the advance of the research areas.