Learning IV&V Strategies

Modern business practices are complex. Consider, for example, NASA's software IV&V (independent verification and validation) team that monitors a diverse range of complex software written by a wide range of contractors from around the world. In an effort to better understand the core business of IV&V, the authors recently conducted DELPHI sessions with experienced IV&V analysts to build a model reflecting their understanding of what level of IV&V is appropriate for different projects. The resulting model, while short, contains subtle interactions that are not immediately apparent. To understand those interactions, we conducted Monte Carlo studies to grow data sets from the model. These data sets where summarized using TAR3 (a minimal contrast set learner) to discover (a) the core business decisions that decide what level of IV&V is appropriate; and (b) whether or not specializations of the problem domain can lead to more simple and robust models.