Pairwise Testing in Real World Practical Extensions to Test Case Generators

Pairwise testing has become an indispensable tool in a software tester’s toolbox. The technique has been known for almost twenty years [22] but it is the last five years that we have seen a tremendous increase in its popularity. Information on at least 20 tools that can generate pairwise test cases, have so far been published [1]. Most tools, however, lack practical features necessary for them to be used in industry. This paper pays special attention to usability of the pairwise testing technique. In particular, it does not describe any radically new method of efficient generation of pairwise test suites, a topic that has already been researched extensively, neither does it refer to any specific case studies or results obtained through this method of test case generation. It does focus on ways in which pure pairwise testing approach needs to be modified to become practically applicable and on features tools need to offer to support a tester trying to use pairwise in practice. The paper makes frequent references to PICT, an existing and publicly available tool built on top of a flexible combinatorial test case generation engine, which implements several of the concepts described herein.